PyCharm, Virtualenv and Environment variables

  • 14
  • 54

Hi lovely people,

I started to use PyCharm for developing my Django projects. I'm very impressed about the functionality it has.

But at the moment I'm having one issue. When I try to use environment variables that I have set in the terminal(export DATABASE_ENGINE=mysql) it isn't working:
print os.environ['DB']

Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/admin_raoul/.virtualenv/i-raoul.com/lib/python2.7/UserDict.py", line 23, in __getitem__
raise KeyError(key)
KeyError: 'DB'

When I check it in the terminal it's working fine. I'm guessing that this error is raised because PyCharm is using a different environment.

Please share your knowledge on how to create environment variables to use with os.environ?

Thanks a lot,
Raoul
Answered by Dmitry Filippov
 
Hi Raoul!

It makes sense of how you launch PyCharm.
You know, environment variables are per-process attribute. And they inherits via 'exec' syscall from parent to child process.
So if you added some variables via 'export' into your current shell... They are environment variables only in this process = this shell instance. So, to make them available as environment variables in PyCharm, PyCharm must be launched from the shell mentioned (thus pycharm is child process of shell).
environment variables are visible through /proc/[pid]/environ (they are per-process)
so you can do something like
$ps aux | grep pycharm to get its pid.
and
$cat /proc/[pid]/environ
please check first if environment variables are really in pycharm process)
so if they are there, pycharm becomes parent to python interpreter instance and it must have environment variables inherited.
so print os.environ['DB'] will work fine in case you have DB variable.

for more information on environment variables please check:
$man 7 environ
$man 3 exec
Which OS are you using?
--
Dmitry Jemerov, Chief Technology Officer, JetBrains

Thanks for your reply Dmitry. I'm using Ubuntu 12.10 (64bit).

To provide you with more insight. I have configured an virtuelenv in PyCharm (File>Settings>Project Interpreters>Python Interpreters). To test it I created a new virtualenv with PyCharm, because the first one was imported.

The system variables are stored in a file called .secrets in the root of my project directory. I created and system link (postactivate<>.secrets):
ln -s /home/raoul/workspace/untitled/.secrets /home/raoul/.virtualenv/untitled/bin/postactivate

I stated my virtualenv with workon untitled and I saw that the variable was set:
echo $DB
mysql

Now I start up PyCharm and try the same in the Python Interpreter but I'm still getting the same error message.

Please advise on how to tackle the situation.

Hi Raoul!

It makes sense of how you launch PyCharm.
You know, environment variables are per-process attribute. And they inherits via 'exec' syscall from parent to child process.
So if you added some variables via 'export' into your current shell... They are environment variables only in this process = this shell instance. So, to make them available as environment variables in PyCharm, PyCharm must be launched from the shell mentioned (thus pycharm is child process of shell).
environment variables are visible through /proc/[pid]/environ (they are per-process)
so you can do something like
$ps aux | grep pycharm to get its pid.
and
$cat /proc/[pid]/environ
please check first if environment variables are really in pycharm process)
so if they are there, pycharm becomes parent to python interpreter instance and it must have environment variables inherited.
so print os.environ['DB'] will work fine in case you have DB variable.

for more information on environment variables please check:
$man 7 environ
$man 3 exec
Dmitry Filippov
Product Marketing Manager
Jetbrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"

Thank you very much for explaining this bit to me. I was able to get it working!

Kind regards,
Raoul

Hi!

In case of virtualenv, why not simply add "source .env" into $VIRTUAL_ENV/bin/activate?
Dmitry Filippov
Product Marketing Manager
Jetbrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"

Hi Dmitry, thanks for the quick answer.
Your method works (mine also did), but I'm searching for a smooth and good alternative. Both yours and my previous solution require some manual settings every time a new developer jumps in the project (i.e., setting up their virtualenv AND remembering to make sure .env is exported).

So I came up with a more automatic solution. Before anything else, my Django settings.py file opens the .env file and exports those variables to the current environment. Here's a gist for that: https://gist.github.com/staltz/5525397

Hi Andre!

Don't you think that locating your module into project root, than importing it and calling import_env_vars is much more simple than just do $echo "source .env" >> $VIRTUAL_ENV/bin/activate ? =)
i don't think so)

Check one more way:
in PyCharm go to Run|Edit Configurations enter to your configuration and play with settings... you can add environment variables or execute script before interpreter invocation, set interpreter options and so on. Anyway it is unable to import environment variables from file... if you want it, than please create feature request http://youtrack.jetbrains.com/issues/PY

let me know if it works?
Dmitry Filippov
Product Marketing Manager
Jetbrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"

Hi Dmitry,
`$echo "source .env" >> $VIRTUAL_ENV/bin/activate` is simpler, but manual (requires additional developers to remember to do this step). My gist is more automatic in that sense.

But actually you mentioned a good idea, I think I will try to just edit the run configuration to execute a script before interpreter invocation.

I guess that executing script will not work here as it modifies it's own environment which is destroyed right after the script has finished.

Probably it is possible to tell interpreter to run some script by its own through interpreter parameters before executing main project. Thus interpreter instance will export everything in this session.
Dmitry Filippov
Product Marketing Manager
Jetbrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"

"Probably it is possible to tell interpreter to run some script by its own through interpreter parameters before executing main project."

It's possible, indeed.
"After these path manipulations, an attempt is made to import a module named sitecustomize, which can perform arbitrary site-specific customizations. It is typically created by a system administrator in the site-packages directory. If this import fails with an ImportError exception, it is silently ignored." - http://docs.python.org/3.3/library/site.html

True (I didn't try that out yet). I will stick to my gist then. And will file a feature request.