DISQUS

Peter Herndon · 5 months ago

The mod_wsgi wiki page http://code.google.com/p/modwsgi/wiki/Reloading... has a section on setting up monitoring in daemon mode. No messy touching a file needed, It Just Works (tm).

jnoller · 5 months ago

That example does much the same thing, it watches the timestamps (instead of the hashes) and then kills the wsgi daemon process. 6 of one way, half a dozen of the other. In my case, I prefer my method above given I already had watcher.py done (for my sphinx stuff).

I added a note about it in my post, thank for reminding me

Jean-Francois Roy · 5 months ago

Unfortunately, you will quickly realize that this setup is not viable on Mac OS X. The issue lies in the way the Python sub-processes are created and the fact a forked process can no longer use most Mac OS X frameworks because its Mach ports have all been invalidated.

Many Python modules on Mac OS X make use of frameworks. For example, urllib uses Carbon to read system proxy settings. The moment would use urllib, the Python process will abort().

The only reliable method I have found to run Python applications on Mac OS X is via FastCGI, with the a single threaded (for what its worth....) Python server process launched independently (preferably via launchd).

jnoller · 5 months ago

Hmm? This is working for me just fine - so far my app has not hit any issues with this, but then again, I'm not using a framework build of python, nor am I using urllib.

Edit: I just chucked 10 urllib.urlopen() calls into the views.py of my app and swapped to python 2.5.2 which ships on leopard; and it doesn't crash. Also, if you're right, then multiprocessing (which is a fork()ed process) on OS/X can not use anything which could call a framework API.

Jean-Francois Roy · 5 months ago

You might be getting lucky on it, or actually found a configuration that works (in which case, awesome!). But I've definitely had issues on Leopard with urllib[2] with mod_wsgi which forced me to switch to FCGI.

This merits more investigation I suppose.

jnoller · 5 months ago

I'm interested in what your configuration was that it broke - I imagine Graham Dumpleton (the mod_wsgi maintainer) would be as well.

grahamd · 5 months ago

The issue with __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__ is not a mod_wsgi problem. It is caused by running mod_php in the same Apache and using a PHP module which links in Carbon libraries. One such example is the phpcups.so module which links to CoreFoundation framework. See 'http://groups.google.com/group/modwsgi/browse_thread/thread/fa1ce8c9a9e45db1'.

grahamd · 5 months ago

Sorry if this is a dup. Looks like the system ate my comment because I didn't have an account. Lost the full message I had posted. Rather than type in again what I said, see:

http://groups.google.com/group/modwsgi/browse_t...

The __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__ message is not a mod_wsgi problem but is caused by PHP modules loaded in mod_php into the same Apache.

jnoller · 5 months ago

Nope it didn't eat it - I just moderate all comments

grahamd · 5 months ago

Cool. I don't recollect the account sign up stuff that intruded into the work flow saying that the comment had still been accepted and was sent for moderation.

Chris F · 5 months ago

According to this (http://code.google.com/p/modwsgi/wiki/Installat...) there are issues with 32 vs 64-bit executables. It would certainly be nice if it just worked. "Thinning" does not look very appealing.

I'm anxious for more info.

jnoller · 5 months ago

Chris, I had to do no such thinning of the executables. I ran the
directions above, and it works. The line right above that states "Note: This issue is fixed in mod_wsgi vesion 2.0. It will backported to version 1.4 if such a version needs to be created."

Nat Williams · 5 months ago

using the Apache shipped with Leopard and a Framework install of 2.6, I did have to thin httpd.

jnoller · 5 months ago

Bleargh. I just installed the python.org version of 2.6 and slammed into the "Non Universal MacPython" issue. Back to my self-compiled version. I'll add notes to the post about the various issues.

Did you try forcing apache into 32bit mode with the "Forcing 32 Bit Execution" instructions on the site?

Michael Shepanski · 5 months ago

Nice find with the wsgi_monitor. Otherwise, I believe I already had pretty much the same setup on my Ubuntu box. Nice to see setting up Apache / Django is becoming a little more standard. :)

Brian · 4 months ago

Trying to get this working has gotten me to pretty much hate mod_wsgi. I followed these directions, then had to deal with "Permission denied Errno 13" errors referring to the location of the python egg cache. Once I figured out a workaround for that, the issues immediately shifted to architecture related nonsense. I'm using the same python installation that I use for all of my non-web-related python development, so why are these issues only popping up when I introduce mod_wsgi into the equation? I thought we were all supposed to accept mod_wsgi as our saviour for implementing a "simple to use Apache module" for Python web apps? This is not simple.

One other question: what are the major differences between using mod_wsgi for the dev work and using the django dev server? Is that documented somewhere?

Sorry man. Don't mean to rant on your parade. Good post, as usual.

jnoller · 4 months ago

Hey Brian-

I have no idea what you're talking about with regards to the python egg cache. I also didn't run into architecture related issues; so I'd be interested to hear what those are. Are you hand compiling it? Is it the built in version? I know Graham has outlined some of the issues here: http://code.google.com/p/modwsgi/wiki/Installat... - the biggest one is the fact the python.org versions of python don't ship 64 bit compatible.

Frankly, most of the issues I've run into are problems with the multi-architecture mess which is leopard+python builds - not mod_wsgi, mod_wsgi just shows the problems. Having a 64 bit apache, and a 32 bit version of Python (which python.org ships) doesn't exactly make things easy for mod_wsgi which has to work with both.

As for the dev vs. production issue - I run the same config on the server (+ or - minus some tweaks) as I do on the desktop.

I can understand your pain, which is why I tried to be explicit as possible. Have you talked to Graham (the mod_wsgi maintainer) about any of this?

Brian · 4 months ago

re: egg cache -- after firing things up the first time, I got an error that ended like this:

[Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] The following error occurred while trying to extract file(s) to the Python egg
[Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] cache:
[Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1]
[Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] [Errno 13] Permission denied: '"'
[Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1]
[Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] The Python egg cache directory is currently set to:
[Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1]
[Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] "/.python-eggs/"
[Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1]
[Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] Perhaps your account does not have write access to this directory? You can
[Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] change the cache directory by setting the PYTHON_EGG_CACHE environment
[Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] variable to point to an accessible directory.

I fixed that by adding a "user=myuser" arg to the WSGIDaemonProcess directive (this after also creating 'chmod 777' dirs owned by the web user, which didn't work, and which I still don't understand). I also tried fixing this using os.environ('PYTHON_EGG_CACHE') in my myapp.wsgi script, and also using the WSGIPythonEggs directive mentioned here: http://code.google.com/p/modwsgi/wiki/Applicati... Nothing worked until I found "user=". BTW, I'm using mod_wsgi 2.5, installed (and dl'd for that matter) using your exact commands, because I was pretty sure going into it that we had near-identical macbook pros.

I'm using the Apple-supplied python installation. I initially created my virtualenv using your instructions exactly, but went back to add '--no-site-packages' when I started having issues loading mysqldb. Then I actually just uninstalled mysqldb from the system, did the "export ARCHFLAGS" like the modwsgi site says to do to get around arch issues, built a new version of the module from source, and now I'm getting:
"[Sun Aug 16 13:23:49 2009] [error] [client 127.0.0.1] raise ImproperlyConfigured("Error loading MySQLdb module: %s" % e)
[Sun Aug 16 13:23:49 2009] [error] [client 127.0.0.1] ImproperlyConfigured: Error loading MySQLdb module: dynamic module does not define init function (init_mysql)
"
I guess it's progress, but again, hardly simple. I haven't finished googling the issue to see what the fix is. It's v. 1.2.3c1 of the module. I guess it's true that this is perhaps mostly an arch-level issue outside the control of modwsgi, but why is it that I can write CLI scripts with Python all day long and never run into any of these issues? I'm not blaming modwsgi or questioning what you've said, I really just want to understand this more clearly, because it's driving me insane, and that's not enjoyable.

I haven't given much thought to arch up to now. Perhaps it'd be easier to just build *everything* from source: apache, python, etc. Have a bare-bones base python install, and just easy_install modules in virtualenv as needed. Advice/opinion hereby solicited. I'm not sure how to reach Graham - I'm sure there's a mod_wsgi google group, so I'll see what I find.

grahamd · 3 months ago

WSGIPythonEggs directive only applies to embedded mode and you are using daemon mode. For daemon mode you use python-eggs option to WSGIDaemonProcess. This is explicitly mentioned in the available documentation for the configuration directives. You can also set os.environ, but either way, Apache normally runs as a special user and so is still going to need write access to the directory you specify. As you find, just easier to use daemon mode and set the user as yourself.

The whole reason the 32/64 bit issues comes up is because 'python' command line program is only installed as 32 bit and so everything runs as 32 bit and when people are using it initially don't see any problems. The MacOS X version of Apache however is a 64 bit capable application and so will default to running 64 bit if you have 64 bit cpu. The distutils package in Python though only builds 32 bit extensions, so unless third party packages have been made MacOS X aware in some way to get around that problem, they will fail to build for all architectures. Some times the simplest thing to do is to install your own version of Apache from source code, as it will only be 32 bit and thus avoid all the problems.

Pretty well all the issues are documented some where as is a reference to mailing list for mod_wsgi. Of course, getting people to read documentation and asking questions on the correct forum is the challenge. :-)

jnoller · 3 months ago

By the way Graham; setting apache/python/mod_wsgi up with the default python2.6 interpreter was dirt simple.