DISQUS

masklinn · 1 year ago

> I doubt they envy the syntax or complex nature of it

What syntax wouldn't there be to envy, and which part of Erlang's concurrency model is complex?

First point: Erlang's concurrency requires 2 primitives: a "send message to process" and a "retrieve message from mailbox". The first one is straightforward and trivial, the second one is a bit more complex (it needs the ability to have timeouts, and while pattern matching makes filtering/fetching messages extremely straightforward it may be more complex to handle that without PM). The send/receive syntax is one of the best parts of erlang's syntax: straightforward, short and to the point. What isn't there to envy? Yes we could add `spawn`, but that's not specific to erlang-style concurrency.

Second point: the complexity. In all of my experiments, Erlang-style concurrency was infinitely simpler than e.g. java-style concurrency, even with java.util.concurrent (1.5's, we haven't switched to 1.6 yet) for various reasons:

* A single concurrency structure (which can be built upon to e.g. supervisor tree, but at its core Erlang only has "send a message, receive a message")

* Unconstrained resources, there is no need to bother with thread-pooling and other tripe/implementation details in Erlang. If you need a process, just spawn it and let the runtime take care of its allocation and scheduling

* Synchronous/sequential erlang and asynchronous/concurrent erlang are clearly separated with function calls on one side and messages on the other one, message sends don't look like function calls and reciprocally.

* Finally, erlang has the tools to handle massive concurrency, in order to introspect and debug a system that can have thousands of running processes.

So I'd like to know more about what you consider to be the syntactical or complexity problems of shared-nothing message-passing concurrency *compared to shared-memory "java-style"* concurrency.

jnoller · 1 year ago

It's not the implementation of Erlang that I'm critiquing - I'm very much pro-shared-little/nothing, ergo my liking of Adam Olsen's monitor work in the safethreading project. I have a bone to pick with erlang's syntax itself.

I agree with you on the power of Erlang's model - however I think the barrier-to-entry of Erlang is why it continues to largely be a niche language. If Python were to adopt some of the theory behind what Erlang is done, I would be happy.

qwerty · 1 year ago

Although pyprocessing isn't the end-all solution, it's syntax is really easy to pick up. I really hope you manage to get it into the standard library.

jnoller · 1 year ago

So do I - and I agree, it's not the final solution, it's only a library to help people get "a" (not The) job done.

MichaelSparks · 1 year ago

Hi there,

You may find Kamaelia interesting - I've been adding clean multicore support recently. You can find an overview of its design philosophy on our summer of code page here: http://kamaelia.sourceforge.net/SummerOfCode2008 (linking there because there's an embedded presentation & lots of links).

You can find discussion of how multicore support was added here:
http://yeoldeclue.com/cgi-bin/blog/blog.cgi?rm=...
Rather than pyprocessing it uses pprocess since that seems sufficient. I'll take a look at pyprocessing though.

Single process pipeline:
Pipeline(
Textbox(position=(20, 340),
text_height=36,
screen_width=900,
screen_height=400,
background_color=(130,0,70),
text_color=(255,255,255)),
TextDisplayer(position=(20, 90),
text_height=36,
screen_width=400,
screen_height=540,
background_color=(130,0,70),
text_color=(255,255,255))
)

Multiprocess pipeline:
ProcessPipeline(
Textbox(position=(20, 340),
text_height=36,
screen_width=900,
screen_height=400,
background_color=(130,0,70),
text_color=(255,255,255)),
TextDisplayer(position=(20, 90),
text_height=36,
screen_width=400,
screen_height=540,
background_color=(130,0,70),
text_color=(255,255,255))
)


It just works. It seems to have similar inspiration as erlang, but I wasn't aware of erlangs execution model when I created Kamaelia. (It was more inspired by asynchronous hardware systems, unix pipelines, CSP and biological systems).

cf http://kamaelia.sourceforge.net/Introduction

I went into that (due to work modelling biological systems) with the assumption that whilst the primary communication would be akin to the nervous system (send and receive of pulses of information between processors) hence why the core of kamaelia is called Axon, that there would need to be an equivalent for a high latency, low use, but useful hormonal system. That boils down to a global key value store. (which will be migrating soon to an STM based store).

The interesting thing from my perspective is that this also mirrors a system from RAF Malvern from 30 years ago, called MASCOT which I heard about at christmas. That uses channels, instead of named inboxes/outboxes, and pools which are equivalent to our CAT ("hormonal" systems). MASCOT is utterly fascinating because they started from the same premise. I only recently heard about MASCOT (just before christmas), and the only reference I can find online is this, but if you scroll down, you'll find "The Official Handbook of Mascot : Version 3.1 : June 1987". Perhaps scarily, their first version appears to have been in 1975....)

http://async.org.uk/Hugo.Simpson/

OK, I'll get back to suggesting ideas for students for GSOC which are naturally highly concurrent, fun, in python beginner friendly and naturally multicore.

Incidentally, there is also another difference between python & concurrency in a functional language - the fact that objects in python are mutable. I blogged about some implications of this here:
http://yeoldeclue.com/cgi-bin/blog/blog.cgi?rm=...

Kamaelia has proof of concept implementations in Java, C++ and ruby as well.

Have fun :-)

Michael.

Joseph Lisee · 1 year ago

Thank you for speaking sense about concurrency in Python. The project hosted at the website listed with this post would benefit in a large way from a CPython without a GIL. I am glad someone else shares the view that python should give you tools to build your application, your way, not dictate its design.

The code for the robotics system above uses more C++ code then needed because it needs true concurrency. A python without a GIL would let us coders benefit from the reduction in code size you get when you go from C++ to python.

-Joseph Lisee

jnoller · 1 year ago

The biggest thing to remember about the GIL is that is it not a boogeyman, but rather an implementation detail of the interpreter. It's design is such that it keeps the C implementation simple especially for extension writers.

Removing the GIL, if done poorly, will grossly increase the complexity of the interpreter, slow C development down and make extension writing a nightmare.

It's important to realize the benefits it does provide.

jnoller · 1 year ago

Kamaelia *does* look interesting. I am going to have to check it out.