jessenoller.com comments - Latest Comments

Re: multiprocessing.Pool and KeyboardInterrupt

Joe — Wed, 17 Jun 2009 06:52:24 -0000

If you change range(2) to range(200) it does not exit correctly. How would you solve this?

Re: SSH Programming with Paramiko | Completely Different

jnoller — Tue, 16 Jun 2009 09:28:18 -0000

Thanks, it looks pretty cool at first glance - it's GPL 3 so I probably won't touch it, but I'll pass it on to other people

Re: SSH Programming with Paramiko | Completely Different

riskable — Tue, 16 Jun 2009 09:23:28 -0000

I know it has been a while since this was posted but I thought I'd follow up and say that thanks to this example and my subsequent discovery (that I posted above) I was able to put together the SSH Power Tool (SSHPT). It is hosted at Google Code:

http://code.google.com/p/sshpt/

In the source code is a real-life usage of sudo over Paramiko (works great in production).

-Riskable
http://riskable.com
"Windows admins click away at the branches of evil.
Linux admins hack at the root $."

Re: YAML ain’t Markup Language | Completely Different

Doug Farrell — Mon, 08 Jun 2009 17:10:56 -0000

Hi Jessie,
Very nice write-up about Yaml, which I'm using. Based on your article in Python Magazine, I started using the syntax you also show above, creating Python objects from the configuration file. I'm even passing them arguments from the Yaml file as you've also shown above.

I do have one question for you, I'd like to put all my configuration files together into one big file. Is there a way to get Yaml to only read one section (or one document)? What I'm trying to avoid is having all the objects constructed by programs that don't care about or use those objects when reading their own configuration section.

Thanks in advance!
Doug

Re: A short list of things I don’t like about Python

Russ Nelson — Fri, 05 Jun 2009 10:05:29 -0000

Introspection is the word you're looking for, not self-awareness.

Re: Twisted - hello, asynchronous programming

Senthil — Wed, 03 Jun 2009 10:09:22 -0000

The fourth link in the reference links should actually be: http://mumak.net/stuff/twisted-intro.html

Re: A short list of things I don’t like about Python

John Cheng — Sun, 31 May 2009 22:12:26 -0000

I've never heard of Rope before. I'll have to add it to my todo list!

Re: A short list of things I don’t like about Python

Collin Winter — Sun, 31 May 2009 15:42:49 -0000

I recall one of the topics of discussion at the PyCon Language Summit (and I'm recalling now, at dinner some days later) was the desire to get rid of the current python.org site architecture and replace it with something easier to administer. The barrier to contribution to python.org is extremely high, even for those of us with commit access. Even a wiki-like system with editing abilities locked down Google code-style would be a massive improvement over the status quo.

What ever became of that discussion? I seem to recall Jacob Kaplan-Moss being interested in actually implementing it. I should ping him...

Re: A short list of things I don’t like about Python

jnoller — Sun, 31 May 2009 14:39:37 -0000

I agree; it's good to discuss, and no - we don't need to agree! That's the beauty of things.

1> Yup. I know you didn't single licensing out; it's a sensitive topic for me, as I obviously fall in the more-permissive-is-good-and-helps-me-put-bread-on-the-table group. Maybe you're right; people will be more consumers than collaborators in a more permissive model, but as I said before, I wouldn't trade that for anything. You have to pick one, and take the good with the bad. *shrug* Sure, there's been plenty of GPL-and-the-like projects which are *huge* successes, that's simply undeniable, but I don't think it's impossible to have great success with a more permissive license, see Free/Open/etc BSD for an example.

2> Wikis are a tough thing. I firmly believe the only reason wikipedia is successful is because of stringent guidelines, and a massive staff of editors constantly patrolling and ensuring quality and factual information. Yes, wikis are good for a low-barrier-to-add, but that's *also* the drawback. A low barrier means a need for constant policing. I'm not saying I don't want people to collaborate, but how much of the information there is of good, high quality?

3> I like your ideas around cleaning it up, and making it a better user experience. I'd also like to add that adding links into the python documentation which references wiki pages or something which says "see wiki topics on xxxx" would make things a lot better too. For example a link in the multiprocessing documentation that says "see addition information at wiki/python_version/multiprocessing/" or wiki/multiprocessing" where the top-level multiprocessing page had information about all of the releases, and pointers to more information.

I'm looking forward to Georg and the GSoC project around the sphinx docs - I am definitely one of those people who prefers structured, high quality documentation (ergo us discussing it), but I am definitely pro making it easier to contribute and find additional information.

I don't know if you have the time, or the will - but maybe someone does need to step up and volunteer to be the BDFL (or a period shorter than life) for the python wiki experience, and be the one to unify/make the hard decisions. Sometimes making those decisions makes you unpopular, but having a good, singular view does make for a more unified experience.

Re: A short list of things I don’t like about Python

Paul Boddie — Sun, 31 May 2009 13:28:38 -0000

Jesse, it's great to have this discussion, even though we obviously don't agree on everything!

My remark about permissive licences didn't single them out as the only factor - if you want an example of a project with good momentum and a certain amount of community interaction with the core developers, I suppose you could also consider PostgreSQL - but the kinds of community that form around projects using different styles of licences are often quite different. I have it on good authority that when the licensing of a project becomes more permissive, you do get an influx of people who are more "consumers" of the code than collaborators around the code. Perhaps the people who feel more "comfortable" about a permissive licence are also the people who are less likely to interact with the development community around that code, who possibly don't see the value in contributing, and who don't feel a common sense of ownership of the project.

I note that your feelings about Wiki solutions has appeared more than in one context during this debate: "And I hate wikis, it's impossible to find anything in them, information is spread all over the place, etc." Although the API documentation probably doesn't belong in a Wiki, there isn't anything better for getting people to contribute or for letting them go off and do something that the documentation maintainers didn't expect. Information about alternative solutions for accessing remote services, for example, is precisely the kind of thing Wikis work well at recording and presenting. Various alternative initiatives around the Python documentation seem to have had all the hallmarks of up-front planning and narrow, predetermined methods of contribution which probably create a queue of change suggestions with a bottleneck similar to what we already have, and leave people wanting to expand the documentation (as opposed to change small pieces of it) out in the cold.

I'm one of the people who administers the Python Wiki, and although I'll admit that it could be better organised, the policy so far has been to tread lightly and not be too severe with the editing. If I had a greater say in the python.org "assets", I'd make a refined version of the Wiki front the whole site experience, dropping the dated and inconvenient-to-edit main site with its glacially paced changes and increasing saturation of links to the Wiki. And I'm not really a Wiki advocate - I just remain baffled at the obsession that exists in various quarters that Wiki solutions are "all very well" but there has to be a "proper site" fronting the whole affair with little concrete justification for that position.

Re: A short list of things I don’t like about Python

carljm — Sun, 31 May 2009 06:27:35 -0000

Emacs with Rope, etags, and flymake/pyflakes: Autocompletion? Check. Syntax errors and warnings reported inline as I type? Check. Automated refactoring? Check. Fast navigation? Check. Anything else you'd like?

Re: A short list of things I don’t like about Python

jnoller — Sat, 30 May 2009 22:10:17 -0000

I'll respond per paragraph:

1> I agree. I think it's easy for people to throw rocks at the XML handling then to find someone willing to step up (as Tarek did for disutils) to take something and improve it incrementally over time, and volunteer to be it's shepherd. The stdlib *needs* something for XML, finding someone to do it is another thing entirely.

2> Again; someone needs to step up. To replace something, or even be added to the stdlib, it now has to be "best of breed" (see brett's post on this here: http://sayspy.blogspot.com/2009/05/staleness-of...). As for people ignoring other things that have happened, that's par for the course, and sometimes (not all the time) even if something has happened elsewhere, it may not be the right solution. It happens. And I hate wikis, it's impossible to find anything in them, information is spread all over the place, etc.

3> The barrier to change things is high: you not only have to have a good idea, you have to be willing to fight for that idea. And yeah, I know we're all on borrowed time, and it is demanding, so things that could change, simply don't.

"Python's Neglect" is largely a side effect of this - the core developers are few, people have to be willing to convince many other people that their idea has not only merit, but that the person proposing the idea will be willing to help create, test, document and maintain the thing they are offering. And Python, because it is in widespread use, has to evolve slowly and relatively carefully.

It is a product of a number of things. Low resources (people, time), high barrier (you have to be willing to fight if you believe in it) and resistance to change (we've always been at war with eurasia).

I do not, however, agree that this is somehow a side-effect of the licensing. One of the reasons I contribute, and have been able to convince my employers to let me contribute to python-core is *because* of the permissive licensing. In my discussions with others, I have heard this anecdotal evidence repeated time, and time again.

Python core does nothing to deny participation, except limit commit privileges to a select group of people who have gained an amount of trust. Everything is open; all discussions are a matter of public record, all checkins are public. Anyone can step up and write a pep (even me) and anyone can submit a patch to the tracker which can find it's way in.

I wouldn't trade the openness, and permissive licensing of the code for something which "forced" (ala the GPL) people and companies to give back. While that has worked for other projects, and continues to do so, I and many other people using python get the luxury of using it, and sometimes giving back to it where we might not be able to do so with something with a forced-participation license.

Part of the beauty of it, is that it's good for individuals, and companies. Companies with half a brain (ala Google, companies I have worked for, etc) give back what they're comfortable with, and when they can. They use python to Get Things Done - and that's what Python is about, getting things done, and not adding restrictions or forced-participation clauses when it simply isn't needed. Part of the reason I love it is this fundamental pragmatism.

Would I like it if people/companies contributed more? Yes. Do I wish we had more people in core dev getting paid to give back? Sure. If I won the lotto tomorrow I'd probably start a small group of people whose sole job was to do nothing but improve the hell out of it. But I see no reason to force this.

There's a lot of things at work here, and a long road to hoe to get "big" changes into python-core, but in the immortal words of Jay-Z: "I got 99 problems but licensing ain't one".

Re: A short list of things I don’t like about Python

Paul Boddie — Sat, 30 May 2009 20:41:51 -0000

I think that a few different phenomena are occurring here, and xml and urllib are interesting examples. With xml, it's completely unfashionable to improve these modules: there are more people out there with an investment in making snide remarks about minidom than there are people willing to improve such modules, but despite the presence of alternatives outside the standard library, the improvements would probably need to start with what's already there.

With urllib, on the other hand, there's a need to look around and see what else is available. I believe itools has modules of a similar nature, and there are third-party HTTP libraries, too. So in this case, the inspiration has to come from elsewhere, and perhaps urllib and urllib2 merely need to live on in some kind of compatibility library. The problem here is that there's a small industry in the Python scene in ignoring what people have already done: expect a PEP or two which ignore most of the work out there, despite there probably being a Wiki page on python.org with all the relevant projects listed.

I'd really like to improve the standard library, but given that any previous advice I've offered has been more or less been brushed aside, and given that we all have only a finite amount of time to do interesting stuff, I don't feel that it's rewarding work. And I guess that you know how demanding the work is, too. Maybe this discussion will change my attitude for a short period of time and I'll do something, but I can't help feeling that "Python's Neglect" is a product of a number of factors including but not limited to things like the way Python is "consumed" by its community and the way the permissive licensing encourages such consumption (as opposed to participation).

Re: A short list of things I don’t like about Python

Ivan — Sat, 30 May 2009 03:27:07 -0000

But having a good language with a poor documentation means you will lose a lot of potential users. I was first drawn to PHP because of the excellent docs which have gotten even better in the last couple of years.

I'm just starting with python and it is a little annoying having to google for things because they are hard to find in the manual.

Re: A short list of things I don’t like about Python

paniq — Sat, 30 May 2009 01:49:13 -0000

SUBMIT A PATCH !!! 1

And besides, that's seven points, not five. You are breaking my heart.

No, actually, I pretty much agree on 1, 4 and 7. For the others, I don't have a problem with that, because I'm a reasonable man. George Bernard Shaw wouldn't like me.

Re: A short list of things I don’t like about Python

John Cheng — Fri, 29 May 2009 22:45:01 -0000

What about an IDE? I come from a Java background, but I use Python for our system administration scripts and have been working on a Python + PyQt project as a hobby. I have yet to find something that will convince me to switch from Emacs.

I would like to be able to rename a module or a function and have all the code that references it be updated. I like autocompletion. I'd like my IDE to statically analyze the code (perhaps via pyLint or PyChecker) and give me an overview of warnings and errors in the entire project. I'd like easier navigation throughout code - for example, being able to easily find all the places where a function is referenced and jump to those places.

Of course, this is not the fault of Python. It just happens that Java is the "hot" language out there, and IBM decided to throw millions of dollars behind building an excellent IDE for it. Having said all of that, I do have to say that I've read good things about Eclipse + PyDev since the last time I tried it, so I will give it another shot this weekend.

I would also like to see more "pure Python" modules. Selfishly, I would like to see a pure Python implementation of a crypto library. It certainly would be more convenient to distribute Python applications when you know your code and the libraries you use will simply work on any platform that runs Python.

I think my nitpicks are not so much about things in Python that I'd like to improve, but a reflection of that Python has a smaller community compared to the major players (Java, .Net). I guess it just means I need to do my part to support and advocate for Python if I want to see said improvements :)

Re: A short list of things I don’t like about Python

iestyn — Thu, 28 May 2009 21:02:08 -0000

I have just started with Python. The threading issue concerns me. BUT, after reading about a bit, it only really affects number crunching stuff and not throughput code. I been doing a bit of threading stuff with c# and before that Delphi. You know, threading ain't that great a thing. With both languages you have all that you describe in your perfect world of Static typed, threaded systems that can access all processors. However, scalability of programs have led me to creating systems that interact and co-operate in a grid style. Also static type systems are a nightmare to add plug-in solutions. Have you ever dealt with a god damn Class inheritance tree that you can't change after code goes to production. If you look at the new C# stuff, all that is Python is being introduced into it in versions 3.5 and 4.0. Dynamic Typing, Linq (List comprehensions!!!). The only gripe in my small and limited time in Python is that of GIL being a referenced based thing. But I give it one thing, its fast and it does kill objects in a timely fashion which is more than I can say for C# and Java. By the way keep up the good work on the processor Modules, Its good stuff, I love the ability to create new processors, and not have the underlying VM decide if I should have one or not :)

Re: A short list of things I don’t like about Python

jnoller — Wed, 27 May 2009 15:30:54 -0000

GSOC Project: http://tosh.pl/gminick/gsoc/sphinx/

Re: A short list of things I don’t like about Python

jnoller — Wed, 27 May 2009 15:21:51 -0000

I know - and I actually use libcurl 99% of the time to be honest. There are historic reasons for all of this, but I don't think there's a good reason to maintain the status quo and not improve ;) except of course, time and resources.

Re: A short list of things I don’t like about Python

jnoller — Wed, 27 May 2009 15:20:18 -0000

I pinged georg, and his comment was "something is coming soon", I almost danced a jig

Re: A short list of things I don’t like about Python

Tom Moertel — Wed, 27 May 2009 14:21:20 -0000

> Your code isn't more secure. It's probably safer and that's about it.

I don't think you understood my claims. I'm not saying that just using static types as usual in, say Java or C#, will secure your code as a side effect. Rather, I'm saying that you can encode security information into types and *then* the compile-time checker will prove that the related security properties hold (or do not); that is, the compiler will prove that your code is secure, for whatever definition of "secure" you have encoded.

In short, your code *will* be more secure, and provably so. In fact, the compiler will discharge the proof obligation for you. (Yes, type systems can actually do meaningful work for you.)

For example, you can use a type-based approach to secure your code against XSS vulnerabilities (and a host of related problems).

> My issue though with static typing is the difficulty to evolve your code.

What makes you think that this difficulty is a property of static typing and not of the popular languages that employ static typing -- and do so poorly?

Cheers,
Tom

Re: A short list of things I don’t like about Python

akuchling — Wed, 27 May 2009 13:55:47 -0000

Regarding httplib: Python's libraries were written in the early & mid 1990s when the web was new, so they're written to allow subclassing and customization in order to allow supporting future HTTP versions, since HTTP had gone from 0.x versions to 1.0, and 1.1 was visible in the future. But now the HTTP 1.1 RFC will turn 10 next month and no one is talking seriously about an HTTP/1.2 or 2.0, so today all that flexibility is just adding complication. It's easier to use libcurl or something similar.

Re: A short list of things I don’t like about Python

csantos — Wed, 27 May 2009 13:49:56 -0000

The Numpy community created an online doc editor last year. From the project site: "Pydocweb is a tool for collaboratively editing docstrings and documentation in a Python module via the web, and merging changes made easily back to the sources." More info in:
http://code.google.com/p/pydocweb/
http://scipy.org/Developer_Zone/DocMarathon2008

Re: A short list of things I don’t like about Python

akuchling — Wed, 27 May 2009 13:46:21 -0000

Searching for 'Javascript diff' finds a few implementations; http://snowtide.com/jsdifflib is a partial translation of Python's difflib, for example. It might be possible to implement the 'submit diff' feature purely in the client's web browser. (The diffs wouldn't be against the original reST, but that's probably OK.)

Re: A short list of things I don’t like about Python

Tom Moertel — Wed, 27 May 2009 12:41:48 -0000

> Of course, you can make the argument that good unit tests would protect you the same way.

How? Seriously, try to make that argument and see how far you get before it starts to crumble under your own scrutiny. It's an enlightening exercise.

Cheers,
Tom