This post originated from an RSS feed registered with Agile Buzz
by James Robertson.
Original Post: Resilient
Feed Title: Michael Lucas-Smith
Feed URL: http://www.michaellucassmith.com/site.atom
Feed Description: Smalltalk and my misinterpretations of life
Resilient is all about making embedded systems work nicely. When a customer has a problem they want to connect to it and fix it without doing product recalls.
He told a funny story about a $100k car that lost its speedo.. he turned the car off, then started it again and he could drive once more!
The embedded market is different to desktop users. Cellphone's that have a bug they will destroy the phone and not buy the phone again. There is higher standards for embedded devices.
This is about presenting a new and fast embedded obejct oriented system that provides product serviceability.
For those who don't know Strongtalk is that it was 4-5x faster than any other Smalltalk around. Java arrived, so they decided they wanted to sell to Wallstreet. Over the weekend they decided to make it suck in Java bytecodes instead of Smalltalk bytecodes. Sun bought them.
Finally he got sick of Java and he's come back to Smalltalk.
At the moment the way they make embedded systems are "Fairly close to pathetic". They make C code that is compiled to C binaries and they link that in to the RTOS Kernel. They then make a binary image and put it on the device. If it fails.. repeat until it works.
The development process has to be very quick because some devices need to be out on the market asap.
Because there is not enough space on the device for debugging, they end up getting a very complex external hardware to do the debugging - but that's at the assembler language. Very highly specialised area.
The current way of doing things: Slow development, low productivity, unsafe programming language, no serviceability, very static model. Once you've sent out the bundle of apps in an image, that's it - some satellites can do a flash of a new image over the old one, but that's it.
Gartner (which is to be taken very lightly) believes that software content in devices is doubling every 2 years. They also seem to think that by 2007 it will be impossible to buy a non-wireless device (yeah and we'll all be driving hovering cars).
Solving the problems: Increased reliability, low cost, resource constraints. Dynamic software updates in the field. Real-time capabilities. Rapid development cycles. He poses a question: Is embedded Java the solution?
Answer: real-time in java is no solution. They can't get rapid development time with Java. Doesn't support incremental execution. Quote: "I should know, I've been working with it longer than most and made HotSpot". Virtual machine specification is very complicated. Bytecodes are not designed for speed and compactness. Configuratios for embedded systems too big (CLDC, CDC, more..).
The java embedded system is 1 meg rom, 0.5 meg ram. Resilient is 128k for both rom and ram usage.
So we can do it better: Use of safe dynamic programming language. Increase productivity is connected to the running embedded device program, provide incremental execution. Provide debugging support in the product with on-the-fly software updates.
In their example they're making speakers that use fire-wire. Here's the catch. Something goes wrong, customer plugs the speaker in to the internet and the company debugs it over the internet, fixing the problem and pushing the answer back in to the speaker straight away.
For mobile phones, you can push out new service applications on to the phones dynamically because the phone is already on a persistent network.
In particular, they can use GPRS to debug the software from anywhere.
They can update methods because they're usually under a hundred bytes - therefore an SMS message can be used to update the phone.
Resilient is a Smalltalk system: Object-based, pure object-oriented (Integers jit to real integers), platform-independant (very important for embedded hardware), libraries and device drivers and networking in the Smalltalk (full TCP/IP stack). Unified real-time resource management. It can also run in a hosted environment, so that it can be tested under Linux and Windows.
Then there is the Resilient IDE which connects to a running device/system, supports true incremental execution and provides the full serviceability and visibility that they aimed for.
Their garbage collector is tuned never to hold up the system visibly for a user to give the experience a 'real-time' feel.
Question: Is the resource management a realtime collector or non-realtime collector? Er.. partially (laughs). The GC in the preview is a standard 2-generation marksweep compacting collector. They're making a new one which will work even better.
With the new way of doing it, you only need to push in the VM on the device.. from then on you just upload your programs in bytecode form and you're away.
The device drivers are written in Smalltalk, so you can change device drivers while you're running.
With one device they had so much memory left over they also threw in a web server so that they could connect it to the internet and see what was playing!
For his demos he has a tiny little computer in a box which is embedded linux (a small pengiun sticker is on the side) which is running Resilient.
One thing that you can't do is to have on the fly code generation, because that would require having source code on there - source code takes up too much space! Their environment has a split, the code is on a computer and it computes the optimum way to get the program running on the device - usually compacted bytecodes.
Their UI they began with Smalltalk/X. That wasn't so great, so then they turned Smalltalk/X in to a web server and you could program in a web browser. They needed bells and whistles and they ended up with lots of silly javascript. The environment they needed to use after that they realised they were cornered.
So, they moved to Eclipse because it has plugins for other languages. They've totally customised Eclipse to work with Smalltalk and their Resilient hardware. It has programming support, bytecode compiler, debugging, profiling and inspecting - which is far beyond what your average Eclipse can do.
The VM is portable and simple. 32bit VM, compact object model - 1-word object headers. Scalable object heap.
Because there is such a small head, they have no VM hash value. A class has to make its own hash implementations. That's the compromise of saving space.
New bytecode set for Smalltalk. 20 bytecodes with uniform format. Optimized interpreter with inline caching (as you'd expect from the Strongtalk guys!)
When you have only 20 bytecodes, you end up with 236 bits free - so you can include several instructions in to the same byte!
Question: Where is the inline cache? It's in the method itself so that it's close at hand. It sits in the literal vector. In most Smalltalks there is a separation of the bytecodes and the dynamic data. They just leave a bit of room in the literal vector.
In Resilient, they have policies for memory management especially when you run out of memory. In Java, when you run out of memory all is over.
Changes to Smalltalk (a few twists): Syntax for full classes (this is no problem, it's just a file-out format..). It's all in files, it's all in CVS - because lots of companies do that. They don't have to waste time making their own source code management too.
They also added an atomic test and store statement. Semaphores, etc, are using this to implement them.
Namespaces are classes, so classes can be nested in classes.
Controversial: Blocks can only exist on the stack. Therefore they can only be local to save lots of memory and cpu cycles. Effectively, they are local anonymous methods in this technique.
They removed pool variables and class instance variables. You can live without them (Yeah, I agree. But they are nice at times - could remake them easily enough in Smalltalk)
The resilient code format is this:
classname = superclassname (
| instance viarables |
methodname = ( ^methodCode)
) class (
key: k value: v = (^super new initializeKey: k value: v)
)
So it's self-ish (heh) with the standard () brackets. The syntax is nice but if you had a nice environment, you shouldn't even care.
I think other Smalltalks could easily adopt their syntax approach if they wanted as an alternative (not a replacement) because it really is just another file-out format.
They have special assign syntax myvar ? nil := .... - because you need it in an atomic unit - one bytecode so that it cannot be pre-empted, they have this syntax because if you call isNil it may get pre-empted and the atomic action will be broken.
Question: Is the LIFO block behaviour temporary? No, we actually like it.
The image has a minimum set of classes to provide basic execution behaviour. This gives them a "build up" approach instead of a "strip down" approach. They provide no reflective behaviour on the system itself. Only the programming environment can do that (this is for security) and perform: is not supported either to avoid the deployment issues of missing something and also to avoid security problems. In the future they also want to do type inferencing on the development environment so that the uploaded bytecode can be pre-optimised.
They also have scheduler and device drivers, network libraries, TCP/IP, SLIP, NIC, Firewire, etc.
Only simple integers. Small integers have 30 bits. Large integers have 32 bit values. If they overflow you get an exception, etc. Small embedded systems really only need 32 bit values, especially for device drivers. Real Large Integers doesn't make so much sense at this point in time.
Question: Encryption? No, we don't do that. But feel free to implement it. It's no problem to add a real Large Integer. They just didn't need to attack that yet.
They have a simplified Collection hierarchy, which is very similar to every other Smalltalk but with a couple of nicer new names.
Question: What about changing array sizes? Arrays are fixed sized. Growing ones allocate a new array.
Question: Can you change the shape of a class? Yes, can't right now but it's supposed to.
Question: Are the bytecode compiler and optimiser, etc written in Java? Yes, because it's in Eclipse. They like it because it's cross platform, etc.
Performance: Allocation of block contexts is slow, so they inline their basic controls and the other one is to flattening of the code to avoid block allocation. Interpretation overhead, which is not a huge problem if its Jitted. Slow method invocation with inline compiler - not much space in an embedded device to do that.
Having these slow operations forces the developer to break down abstractions to get speed. They don't want that to happen.
Therefore, you must mark which arguments to a method are blocks so that the compiler knows what to track inside the method and how it is going to be used. because they're tracking blocks they can also make sure they do not escape. You can also not return blocks either. Most code does not return blocks or assign them to things. SortedCollection uses blocks. They can't do this, so you make a subclass of SortedCollection with your sort method on it to get around that.
But by having these restricted blocks they get a 5x speed improvement! The actual Smalltalk code stays the same, there's no syntax change in there. But the method definition is the one that marks the block by putting [ ] around the argument name.
Question: Ruby does something similar with blocks. It has a way to convert a LIFO block in to a block closure. It copies the context off the stack in to a miniature heap. It's probably something you don't need, but could you do that? Yes, you could do that. They decided not to do it to keep things simple.
10% of methods can be eliminated if methods with identical bytecodes are shared. Resilient IDE tracks updated methods and keeps it compacted.
Superbytecodes are combinations of the basic bytecodes. Reduces bytecodes by 45%. They did bytecode histrograms to discover which super bytecodes they need.
Question: How many of the 236 have they used? We're using 35 now, so not very many. Lots more can be done.
OS Layer services in Resilient: Theads, Stacks, Synchronisation, Device Driver API, Memory mapped I/O, Interrupts.
You can get yourself a technology preview from www.oovm.com - version 1.0 will be ready on late summer 2004. You can use it for fun, for free - but if you do it commercially you have to buy a license.
My summary: Real solution to real problems, a nice small tight smalltalk. Very neat, very cool. The one thing that concerns me is using Eclipse to do development. I'd rather see a Smalltalk editing environment for it. Now I have two great options. Resilient for very light weight and VW WindowsCE for more heavy weight devices. Resilient lets me make device drivers in Smalltalk while VW WindowCE is for sitting in a hosted environment.
In the Demo he has an LCD screen, which is a very small chip. He dynamically tells it what to show.. then changes one of the libraries and updates it and the font shrinks. Debugging is still primitive. Already you can see all the threads on the device and see the stack but there doesn't appear to be edit-continue or process interruption at the moment.
Question: Do you support exceptions? But of course!
Oops: He dropped the linux box. "It's resilient"
Exception handling isn't fully implemented. It's enough for now. It just throws symbols around. They will do more to make it compatible with other Smalltalk systems.
Question: Have you considered a port to PalmOS? No. We're focusing on small devices.
Question: is there a secure connection to the device? Not yet, but there will be.
There is a wiki for Resilient.
Question: Why drop Smalltalk/X? Mostly, the programmers we want to convince to use this. They are used to Visual C . It's much easier to convert to an Eclipse sort of environment than a paint-based environment. They say they're not UI's guy, which is why they went with Eclipse. Yeah it was a pain to write it in Java.. but.. eh.. drink some more coffee.
Question: Are you planning to do hardware emulation so that you can try it out without the real device? No. Just use the device. It's all dynamic any way now.
Previous Topic
