Category Archives: Linux

Teensy 3.1 Bare-Metal

Introduction

A couple of weeks ago I saw a link on hackaday to an article by Karl Lunt about using the Teensy 3.1 without the Arduino IDE and building for the bare metal. I was very intrigued as the Arduino IDE was my only major beef with developing stuff for the Teensy 3.1 and I wanted to be able to do things without having to use the IDE. I read through the article and although it was geared towards windows, I decided to try to adapt it to my development style. There were a few things I wanted to do:

  • No additional code dependencies other than the teensyduino installation which I already had
  • Use local binaries for compilation, not the ones included with teensyduino (it just felt uncomfortable to use theirs)
  • Separation of src, obj, and bin directories
  • Mixture of c and cpp files in the src directory
  • Not needing to explicitly list the files in the src directory to compile
  • Selective inclusion of features from the teensyduino installation

I have very little experience writing more complex Makefiles. When I say “complex” I am referring to makefiles which have the src, obj, bin separation and pull in objects from multiple sources. While this may not seem complex to many people, its something I have very little experience actually doing by hand (I would normally use a generator of some sort).

I’m writing this in the hope that those without mad Makefile skills, such as myself, can liberate themselves from the Arduino IDE when developing for an awesome platform like the Teensy 3.1.

All code for this example can be found here: https://github.com/kcuzner/teensy31-blinky-bare-metal

Prerequisites

As my first order of business, I located the arm-none-eabi binaries for my linux distribution. These can also be found for Windows as noted in Karl Lunt’s article. Random sidenote: I found this description of why arm-none-eabi is called arm-none-eabi. Very informative. Anyway, for those who run archlinux, the following packages are needed:

  • arm-none-eabi-gcc (contains the compilers)
  • arm-none-eabi-binutils (contains the linker, objdump, and other things for manipulating the binaries into hex files)
  • make (we are using a makefile…)

Hopefully this gives a bit of a hint on what packages may need to be installed on other systems. For Windows, the compiler is here and make can be found here or by googling around. I haven’t tested any of this on Windows and would advocate using Linux for this, but it shouldn’t be hard to modify the Makefile for Windows.

My Flow

For C and C++ development I have a particular flow that I like to follow. This is heavily influenced by my usage of Code::Blocks and Visual Studio. I like to have a src directory where I put all of my sources, an include directory where I put all of my headers, an obj directory for all the obj, d, & lst files, and a bin directory for my executable output. I’ve always had such a hard time with raw Makefiles because I could never quite get that directory structure working. I was never quite satisfied with my feeble Makefile attempts which ended up placing the object files in the root directory where the sources had to be. This Makefile represents my first time I was ever able to actually have a real bin, obj, src structure that works.

Compiling object files to obj & looking in src for source

A working description of this can be found in the Makefile in my github repository I mentioned earlier.

Makefiles work by defining a series of “targets” which have “dependencies”. Every dependency can also be the name of a target and a target may have multiple ways of being resolved (this I never realized before). So, here is the parts of the Makefile which enable searching in src for both c and cpp and doing specific actions for each, comping them into the obj directory:

Each section above has a specific purpose and the order can be rather important. The first part uses $(wildcard …) to pick up all of the C++ and C files. The CPP_FILES variable, for example, will become “src/file1.cpp src/file2.cpp src/etc.cpp” if we had “file1.cpp”, “file2.cpp” and “etc.cpp” in the src directory. Similarly, the C_FILES would pick up any files in src with a c file extension. Next, the filenames are transformed into object filenames living in the obj directory. This is done by first changing the file extension of the files to .o using the $(CPP_FILES:.cpp=.o) or $(C_FILES:.c=.o) syntax. However, these files still look like they are in the src directory (e.g. src/file1.o) so the directory is next stripped off each file using $(nodir…). Removing the directory doesn’t allow for a nested src directory, but that wasn’t one of our objectives here. At this point, the files are just names with no directories (e.g. file1.o) and so the last step is to change them to live in the obj directory using $(addprefix $(OBJDIR)/,..). This completes our transformation, populating OBJ_FILES to look like “obj/file1.o obj/file2.o” etc.

The next part is where we take that list of object files and use them as dependencies for a target. Targets are defined by <target name>: <dependency list> followed by a list of commands to execute after resolving the dependencies. IMPORTANT: The list of commands needs to be indented by a tab (\t) character. Spaces will not work (it will say something like “missing separator” with a line number). A target is anything that we pass into make. The default target is ‘all’. The “dependencies” are files which much be “up to date” before the target is run.

In our example, we use $(OBJ_FILES) as a dependency of “$(OUTPUTDIR)/$(PROJECT).elf” which is required as a dependency of “build”. This tells make that when we run “make build”, it needs to try to resolve the dependency of “bin/<project>.elf” which in turn needs to resolve “obj/file1.o”, “obj/file2.o”, and “obj/etc.o” (going from our example in the previous paragraph). This is where the next couple targets come in. A target will only be executed if it can find some rule to resolve all of the dependencies. We will use “obj/file1.o” as an example here. There are 2 targets with that name, actually: “$(OBJDIR)/%.o: $(SRCDIR)/%.c” and “$(OBJDIR)/%.o: $(SRCDIR)/%.cpp”. It would be good to note that the target names here the exact same even though the dependencies are different. Now, how does “$(OBJDIR)%.o” match “obj/file1.o”? A Makefile does something called “pattern matching” when the % sign is used. It says “match something that looks like $(OBJDIR)<some file>.o” which our “obj/file1.o” happens to match. The cool part is that once the target name is resolved using a %, the dependencies get to use % to substitute the exact same thing. Thus, our % here is “file1″, so it follows that its dependency must be “$(SRCDIR)/file1.c”. Now, our example used “file1.cpp”, not “file1.c” and this is where defining multiple targets with the same names but different dependencies comes in. A target will only be executed if the dependencies can be resolved to either an actual file and/or another target. Our first target won’t be a match since it says that the source file should be a C file. So, it goes to the next target that matches the name which has a dependency of “$(SRCDIR)/file1.cpp”. This one matches, and so commands following that target are executed.

When executing a target (“$(OBJDIR)/%.o: $(SRCDIR)/%.cpp” in our example), there are some special variables which are available for use. These are described here, but I will discuss two important ones that I used: $@ and $<. $@ is the name of the target (so, “obj/file.o” in our case) and $< is the name of the first dependency (“src/file.cpp” in our case). This lets us pass these arguments into the commands that we execute. Our Makefile will first create the obj directory by calling “mkdir -p $(dir $@)” which is translated into “mkdir -p obj” since $(dir $@) will give us “obj”. Next, we actually compile the $< (which is translated to “src/file.cpp”), outputting it to $< which is translated to “obj/file.o”.

Outputting everything to bin

Compared to the pattern matching and multiple target definitions that we discussed above, this is comparatively simple. We simply get to prefix all of our “binary” output files with some directory which is set as $(OUTPUTDIR) in my Makefile. Here is an example:

We see here that any output that we are creating as a result of the compilation (.elf, .hex, .bin) is going to end up in $(OUTPUTDIR). Futher, we see that our “all” target asks the Makefile to create both a bin file and a hex file along with two other targets called “stats” and “dump”. These are just scripts that execute the “size” and “objdump” commands on our bin file.

Using Teensyduino without compiling everything

This was by far the most frustrating part to get working. Everything about the makefiles was readily available online, with some serious googling. However, getting things to actually compile was a little different story.

The thing that makes this complex is the fact that it seems the Teensyduino libraries were not designed to be used independently of each other. I will cover, in order, what steps I had to take in order to get this to work.

The most important file we need is called “mk20dx128.c”. This sets up a lot of things relating to interrupts along with the Phase Lock Loop (PLL) which controls the speed of the Teensy’s processor. Without this configuration, we don’t get interrupts and the processor runs at a pitiful 16Mhz. The only problem is that “mk20dx128″ references a few functions that are either part of the standard library and not used often (making them difficult to search for) or are defined in other files, increasing our dependency count.

My first mistake was explicitly using the linker to link all of my object files (wait…aren’t we supposed to use the linker? Read on.). Since arm-none-eabi is not dependent on a specific architecture, it doesn’t know which standard library (libc) to use. This results in an undefined reference to “__libc_init_array()”, a function used during the initialization phase of a program which is not often invoked in code outside the standard library itself. mk20dx128.c uses this function in its custom startup code which prepares the processor for running our program. To solve this, I wanted to tell the linker that I was using a cortex-m4 cpu so that it would know which libc to include and thereby resolve the reference. However, this proved difficult to do when directly invoking the linker. Instead, I took a hint from the Makefile that comes with Teensyduino and used the following command to link the objects:

Which more or less translates to (using our example from earlier):

We would have thought that we should be using arm-none-eabi-ld instead of arm-none-eabi-gcc. However, by using arm-non-eabi-gcc I was able to pass the argument “-mcpu=cortex-m4″ which then allowed GCC to instruct the linker which standard library to use. Wonderful, right? So all of our problems are solved? Not yet.

The next thing is that mk20dx128.c has a lot of external dependencies. It uses a function defined in pins_teensy.c which in turn requires functions defined in both analog.c and usb_dev.c which opens another can of worms. Ugh. I didn’t want this many dependencies and I couldn’t see a way to escape compiling nearly the entire Teensyduino library just to run my simple blinking program. Then, it dawned on me: I could use the same technique that mk20dx128.c uses to define its ISRs to “define” the functions that pins_teensy.c was calling that I didn’t really want. So, I made a file called “shim.c” which contained the following:

I decided that I would include “yield.c” and “analog.c” since those weren’t too big. This left just the usb stuff. The only function that was actually called from pins_teensy.c was “usb_init”. What the above statement says to the compiler is “I am defining usb_init(void) here (which points to unused_void(void)) unless you find another definition of usb_init(void) somewhere”. The “weak” attribute makes this “strong” symbol of usb_init a “weak” symbol reference to which is basically the same as just making a declaration (in contrast to the definition a function, which is usually a strong reference). Sidenote: A program can have any number of weak symbol references to a specific function/variable (declarations), but only one strong symbol reference (definition) of that function/variable. The “alias” attribute allows us to say “when I say usb_init I really mean unused_void”. The end result of this is that if nobody defines usb_init(void) anywhere, as would be situation if I were to decide not to include usb_dev.c, any calls to usb_init(void) will actually call unused_void(void). However, if somebody did define usb_init(void), my definition of usb_init would be ignored in favor of using their definition. This lets me include usb support in the future if I wanted to. Isn’t that cool? That fixed all of my reference issues and let me actually build the project.

Conclusion

Armed with my new Makefile and a better understanding of how the Teensy 3.1 works from a software perspective, I managed to compile and upload my “blinky” program which just blinks the onboard LED (pin 13) on and off every 1/4 second. The overall program size was 3% of the total space, which is much more reasonable compared to the 10-20% it was taking when compiled using the Arduino IDE.

Again, all files from this escapade can be found here: https://github.com/kcuzner/teensy31-blinky-bare-metal

First thoughts on the Teensy 3.1

Wow it has been a while; I have not written since August.

I entered a contest of sorts this past week which involves building an autonomous turret which uses an ultrasonic sensor to locate a target within 10 feet and fire at it with a tiny dart gun. The entire assembly is to be mounted on servos. This is something my University is doing as an extra-curricular for engineers and so when a friend of mine asked if I wanted to join forces with him and conquer, I readily agreed.

The most interesting part to me, by far, is the processor to be used. It is going to be a Teensy 3.1:

Teensy 3.1

Teensy 3.1

This board contains a Freescale ARM Cortex-M4 microcontroller along with a smaller non-user-programmable microcontroller for assistance in the USB bootloading process (the exact details of that interaction are mostly unknown to me at the moment). I have never used an ARM microcontroller before and never a microcontroller with as many peripherals as this one has. The datasheet is 1200 pages long and is not really even being very verbose in my opinion. It could easily be 3000 pages if they included the level of detail usually included in AVR and PIC datasheets (code examples, etc). The processor runs at 96Mhz as well, making it the most powerful embedded computer I have used aside from my Raspberry Pi.

The Teensy 3.1 is Arduino-compliant and is designed that way. However, it can also be used without the Arduino software. I have not used an Arduino before since I rather enjoy using microcontrollers in a bare-bones fashion. However, it is become increasingly more difficult for me to be able to experiment with the latest in microcontroller developments using breadboards since the packages are becoming increasingly more surface mount.

The Arduino IDE

Oh my goodness. Worst ever. Ok, not really, but I really have a hard time justifying using it other than the fact that it makes downloading to the Teensy really easy. This post isn’t meant to be a review of the arduino IDE, but the editor could use some serious improvements IMHO:

  • Tab indentation level: Some of us would like to use something other than 2 spaces, thank you very much. We don’t live in the 70′s where horizontal space is at a premium and I prefer 4 spaces. Purely personal preference, but I feel like the option should be there
  • Ability to reload files: The inability to reload the files and the fact that it seems to compile from a cache rather than from the file itself makes the arduino IDE basically incompatible with git or any other source control system. This is a serious problem, in my opinion, and requires me to restart the editor frequently whenever I check out a different branch.
  • Real project files: I understand the aim for simplicity here, but when you have a chip with 256Kb of flash on it, your program is not going to be 100 lines and fit into one file. At the moment, the editor just takes everything in the directory and compiles it by file extension. No subdirectories and every file will be displayed as a separate tab with no way to close it. I am in the habit of separating my source and not having the ability to structure my files how I please really makes me feel hampered. To make matters worse, the IDE saves the original sketch file (which is just a cpp file that will be run through their preprocessor) with its own special file extension (*.ino) which makes it look like it should be a project file, but in reality it is not.

There are few things I do like, however. I do like their library of things that make working with this new and foreign processor rather easy. I also like that their build system is very cross-platform and easy to use.

First impression of the processor

I must first say that the level of work that has gone into the surrounding software (the header files, the teensy loader, etc) truly shows and makes it a good experience to use the Teensy, even if the Arduino IDE sucks. I tried a Makefile approach using Code::Blocks, but it was difficult for me to get it to compile cross-platform and I was afraid that I would accidentally overwrite some bootloader code that I hadn’t known about. So, I ended up just going with the Ardiuno IDE for safety reasons.

The peripherals on this processor are many and it is hard at times to figure out basic functions, such as the GPIO. The manual for the peripherals is in the neighborhood of 60 chapters long, with each chapter describing a peripheral. So far, I have messed with just the GPIOs and pin interrupts, but I plan on moving on to the timer module very soon. This project likely won’t require the DMA or the variety of onboard bus modules (CAN, I2C, SPI, USB, etc), but in the future I hope to have a Teensy of my own to experiment on. The sheer number of registers combined with the 32-bit width of everything is a totally new experience for me. Combine that with the fact that I don’t have to worry as much about the overhead of using certain C constructs (struct and function pointers for example) and I am super duper excited about this processor. Tack on the stuff that PJRC created for using the Teensy such as the nice header files and the overall compatibility with some Arduino libraries, and I have had an easier time getting this thing to work than with most of my other projects I have done.

Conclusion

Although the Teensy is for a specific contest project right now, at the price of $19.80 for the amount of power that it gives, I believe I will buy one for myself to mess around with. I am looking forward to getting more familiar with this processor and although I resent the IDE I have to work with at the moment, I hope that I will be able to move along to better compilation options that will let me move away from the arduino IDE.

Raspberry Pi as an AVR Programmer

Introduction

Recently, I got my hands on a Raspberry Pi and one of the first things I wanted to do with it was to turn it into my complete AVR development environment. As part of that I wanted to make avrdude be able to program an AVR directly from the Raspberry Pi with no programmer. I know there is this linuxgpio programmer type that was recently added, but it is so recent that it isn’t yet included in the repos and it also requires a compile-time option to enable it. I noticed that the Raspberry Pi happens to expose its SPI interface on its expansion header and so I thought to myself, “Why not use this thing instead of bitbanging GPIOs? Wouldn’t that be more efficient?” Thus, I began to decipher the avrdude code and write my addition. My hope is that things like this will allow the Raspberry Pi to be used to explore further embedded development for those who want to get into microcontrollers, but blew all their money on the Raspberry Pi. Also, in keeping with the purpose that the Raspberry Pi was originally designed for, using it like this makes it fairly simple for people in educational surroundings to expand into different aspects of small computer and embedded device programming.

As my addition to avrdude, I created a new programmer type called “linuxspi” which uses the userspace SPI drivers available since around Linux ~2.6 or so to talk to a programmer. It also requires an additional GPIO to operate as the reset. My initial thought was to use the chip select as the reset output, but sadly, the documentation for the SPI functions mentioned that the chip enable line is only held low so long as the transaction is going. While I guess I could compress all the transactions avrdude makes into one giant burst of data, this would be very error prone and isn’t compatible with avrdude’s program structure. So, the GPIO route was chosen. It just uses the sysfs endpoints found in /sys/class/gpio to manipulate a GPIO chosen in avrdude.conf into either being in a hi-z input state or an output low state. This way, the reset can be connected via a resistor to Vcc and then the Raspberry Pi just holds reset down when it needs to program the device. Another consequence which I will mention here of choosing to use the Linux SPI drivers is that this should actually be compatible with any Linux-based device that exposes its SPI or has an AVR connected to the SPI; not just the Raspberry Pi.

Programming an AVR

Raspberry Pi Programming an AVR

Usage

So, down to the nitty gritty: How can I use it? Well, at the moment it is in a github repository at https://github.com/kcuzner/avrdude. As with any project that uses the expansion header on the Raspberry Pi, there is a risk that a mistake could cause your Raspberry Pi to die (or let out the magic smoke, so to speak). I assume no responsibility for any damage that may occur as a result of following these directions or using my addition to avrdude. Just be careful when doing anything involving hooking stuff up to the expansion port and use common sense. Remember to measure twice and cut once. So, with that out of the way, I will proceed to outline here the basic steps for installation and usage.

Installation

The best option here until I bother creating packages for it is to  do a git clone directly into a directory on the Raspberry Pi and build it from there on the Raspberry Pi itself. I remember having to install the following packages to get it to compile (If I missed any, let me know):

  • bison
  • autoconf
  • make
  • gcc
  • flex

Also, if your system doesn’t have a header at “linux/spi/spidev.h” in your path, you probably need to install that driver. I was using Arch Linux and it already had the driver there, so for all I know its always installed. You also should take a look to make sure that “/dev/spidev0.0″ and “/dev/spidev0.1″ or something like that exist. Those are the sort of endpoints that are to be used with this. If they do not exist, try executing a “sudo modprobe spi_bcm2708″. If the endpoints still aren’t there after that, then SPI support probably isn’t installed or enabled for your kernel.

After cloning the repo and installing those packages, run the “./boostrap” script which is found in the avrdude directory. This will run all the autoconf things and create the build scripts. The next step is to run “./configure” and wait for it to complete. After the configure script, it should say whether or not “linuxspi” is enabled or disabled. If it is disabled, it was not able to find the header I mentioned before. Then run “make” and wait for it to complete. Remember that the Raspberry Pi is a single core ARM processor and so building may take a while. Afterwards, simply do “sudo make install” and you will magically have avrdude installed on your computer in /usr/local. It would probably be worthwhile to note here that you probably want to uninstall any avrdude you may have had installed previously either manually or through a package manager. The one here is built on top of the latest version (as of May 26th, 2013), so it should work quite well and be all up to date and stuff for just using it like a normal avrdude. I made no changes to any of the programmer types other than the one I added.

To check to see if the avrdude you have is the right one, you should see an output similar to the following if you run this command (tiny-tim is the name of my Raspberry Pi until I think of something better):

Note that right under “linuxgpio” there is now a “linuxspi” driver. If it says “(not available)” after the “linuxspi” description, “./configure” was not able to find the “linux/spi/spidev.h” file and did not compile the linuxspi programmer into avrdude.

Configuration

There is a little bit of configuration that happens here on the Raspberry Pi side before proceeding to wiring it up. You must now decide which GPIO to sacrifice to be the reset pin. I chose 25 because it is next to the normal chip enable pins, but it doesn’t matter which you choose. To change which pin is to be used, you need to edit “/usr/local/etc/avrdude.conf” (it will be just “/etc/avrdude.conf” if it wasn’t built and installed manually like above). Find the section of the file that looks like so:

The “reset = ” line needs to be changed to have the number of the GPIO that you have decided to turn into the reset pin for the programmer. The default is 25, but that’s just because of my selfishness in not wanting to set it to something more generic and having to then edit the file every time I re-installed avrdude. Perhaps a better default would be “0″ since that will cause the programmer to say that it hasn’t been set up yet.

Wiring

After setting up avrdude.conf to your desired configuration, you can now connect the appropriate wires from your Raspberry Pi’s header to your microchip. A word of extreme caution: The Raspberry Pi’s GPIOs are NOT 5V tolerant, and that includes the SPI pins. You must do either one of two things: a) Run the AVR and everything around it at 3.3V so that you never see 5V on ANY of the Raspberry Pi pins at any time (including after programming is completed and the device is running) or b) Use a level translator between the AVR and the SPI. I happen to have a level translator lying around (its a fun little TSSOP I soldered to a breakout board a few years back), but I decided to go the 3.3V route since I was trying to get this thing to work. If you have not ever had to hook up in-circuit serial programming to your AVR before, perhaps this would be a great time to learn. You need to consult the datasheet for your AVR and find the pins named RESET (bar above it), MOSI, MISO, and SCK. These 4 pins are connected so that RESET goes to your GPIO with a pullup resistor to the Vcc on your AVR, MOSI goes to the similarly named MOSI on the Raspberry Pi header, MISO goes to the like-named pin on the header, and SCK goes to the SPI clock pin (named SCLK on the diagram on elinux.org). After doing this and double checking to make sure 5V will never be present to the Raspberry Pi, you can power on your AVR and it should be able to be programmed through avrdude. Here is a demonstration of me loading a simple test program I made that flashes the PORTD LEDs:

There are two major things to note here:

  • I set the programmer type (-c option) to be “linuxspi”. This tells avrdude to use my addition as the programming interface
  • I set the port (-P option) to be “/dev/spidev0.0″. On my Raspberry Pi, this maps to the SPI bus using CE0 as the chip select. Although we don’t actually use CE0 to connect to the AVR, it still gets used by the spidev interface and will toggle several times during normal avrdude operation. Your exact configuration may end up being different, but this is more or less how the SPI should be set. If the thing you point to isn’t an SPI device, avrdude should fail with a bunch of messages saying that it couldn’t send an SPI message.

Other than that, usage is pretty straightforward and should be the same as if you were using any other programmer type.

Future

As issues crop up, I hope to add improvements like changing the clock frequency and maybe someday adding TPI support (not sure if necessary since this is using the dedicated SPI and as far as I know, TPI doesn’t use SPI).

I hope that those using this can find it helpful in their fun and games with the Raspberry Pi. If there are any issues compiling and stuff, either open an issue on github or mention it in the comments here.

256Mb doesn’t do what it used to…

So, this last week I got an email from rackspace saying that my server was thrashing the hard drives and lowering performance of everyone else on that same machine. In consequence, they had rebooted my server for me.

I made a few mistakes in the setup of this first iteration of my server: I didn’t restart it after kernel updates, I ran folding@home on it while running nodejs, and I didn’t have backups turned on. I had it running for well over 200 days without a reboot while there had been a dozen or so kernel updates. When they hard rebooted my server, it wouldn’t respond at all to pings, ssh, or otherwise. In fact, it behaved like the firewalls were shut (hanging on the “waiting” step rather than saying “connection refused”). I ended up having to go into their handy rescue mode and copy out all the files. I only copied my www directory and the mysql binary table files, but as you can see, I was able to restore the server from those.

This gave me an excellent opportunity to actually set up my server correctly. I no longer have to be root to edit my website files (yay!), I have virtual hosts set up in a fashion that makes sense and actually works, and overall performance seems to be improved. From now on, I will be doing the updates less frequently and when I do I will be rebooting the machine. That should fix the problem with breaking everything if a hard reboot happens.

I do pay for the hosting for this, 1.5 cents per hour per 256Mb of RAM with extra for bandwidth. I only have 256Mb and since I don’t make any profit off this server whatsoever at the moment, I plan on keeping it that way for now. Considering that back in the day, 256Mb was a ton of memory, it clearly no longer suffices for running too much on my server (httpd + mysql + nodejs + folding@home = crash and burn).

MMO Asteroids in Node.js…but for real.

Many people saw this april fools joke where the author said that he had created an asteroids MMO using Node.js. In reality, it was completely client side and was a bunch of bots. However, I did find the whole thing rather intriguing and decided to see what I could do with Node.js along this line. Last week I started on the project and this weekend I made enough progress that I can publish v0.0.1. It leaves several things to be desired, including a better game background so that one can tell when the view is shifting and user tracking so that you accumulate your scores over time. It does work and from what I can see its not horribly bad performance. While it certainly won’t be able to handle thousands of clients, I expect that it should be able to handle somewhere between 50-100 before it starts dying. At the moment, its quite limited by the memory on my server and the client side scripting gives the impression of “stuttering” with the dead reckoning system used to make the animations smooth. The stuttering is caused by the linear and angular damping that I have running on the server side not being factored into the projected location on the client side.

For physics I am using Box2Dweb in a Node.js module which may be overkill, but its the simplest Javascript physics engine I could find since I didn’t feel like writing my own. The server keeps track of all of the entities in the game and each client requests a “view” of an area of the room. The client is informed which player is them, but other than that it just sees all players and entities in the same list. The actual rendering function draws the entities onto the canvas dependent upon the type of the object.

I have made the source available on github here: https://github.com/kcuzner/mmo-asteroids

The MMO itself can be viewed here: http://kevincuzner.com:8080/. Note that at times it may be down since I am messing with my firewall right now and I might accidentally close the port. Just leave me a note in the comments and I’ll try to get it working.

Simulink clone…now in C++

I took the plunge and decided to re-implement what I had in Python using C++. I had to change up my structure a bit, but I made the switch because of the following reasons:

  • The Qt Framework. While there are bindings for Python, I really liked Qt-Creator. Also, Qt is extremely cross platform (at least that is how it seems) and has a large amount of libraries. I’ve been messing around with it now for a bit trying to get a few things to work.
  • This is redundant of the above, but I really like the Qt Plugin system. After struggling with it for a bit, I finally was able to get a plugin to load. I will explain a little bit below exactly what I plan on doing with these.
  • As awesome as Python is, it doesn’t support true multithreading (it does multiprocessing…and to do what I wanted to do using multiprocessing would have required me to jump through some hula hoops)
  • C++ should have the potential to run faster than Python for mathematical things, which in this situation is a good trade off for ease of programming.
  • Writing the application entirely in C++ with Qt lowers the number of dependencies that would have to be installed on a client machine.

Now, the biggest advantages I see are: Speed and Extensibility. Python is extremely extensible, but it doesn’t have the greatest speed. C++ has the potential to run faster and then by using Qt, the extensibility part was brought in. I also prefer strongly typed systems since they keep me from stepping on my toes programmatically.

By far the coolest part of all this is the Plugins. After discovering that ALL communication between Plugins and the application must be done using interfaces, I realized that if I were to implement the entire thing using interfaces it could be extended to do many awesome things. So far, however, I have only really added interfaces to allow for adding computational “blocks” to the system for use in schematics. The system itself will define no blocks since I have decided to separate the engine from the actual blocks.

I will be posting later a bit about Qt plugins since that is what I have spent the most time on. Google was definitely my friend on that one. Most people it seems just use Qt Plugins for extending Qt itself rather than doing the “low level” extending the application stuff.

In terms of development time, C++ is quite a bit slower for me than Python. However, my potential to write good code is much higher since I am much more familiar with C++ coding conventions and I am more able to clean code while being confident nothing is being broken since in Python, there are no compile-time errors to tell you that you switched the arguments to a function.

Cloning Simulink…in Python

For a while now I have been working on a bench supply. As part of this I have been trying to get a PID controller to work. At first it was simple, but after asking my Dad about it (he does power electronics), he suggested that I use a cascaded PID loop for controlling the voltage and current using the voltage alone. I have sort of a bench model going, but I don’t really want to start construction until I have everything finalized since blowing things up and making mistakes is kind of expensive for my meager college student budget. Tweaking that without a working bench model that I am willing to blow up is kind of hard, so I started trying to figure out how to simulate it. Being partial to simulink (I’ve used it before with some nice pre-built blocks), I wanted to be able to lay it out graphically like control system diagrams usually show and I also wanted to be able to view plots over time. So thus was born my latest project: SimuPy.

Python seemed like an ideal language for this since I wanted it to look nice, be extensible, and be almost universally cross platform. I am relying heavily on the Qt library because it runs on almost anything and it has the ability to use slots and signals on pretty much any object as well. I guess another option would have been Java, but seeing as I don’t like Java that much, Python was what I went with. In addition, I am weighing a couple other options:

  • At the sacrifice of portability, I could make the interface half in C++ and half Python so that I still keep the extensibility without having the requirement of installing python on the person’s computer to get it to work. The simulation part would then turn into a python library that the program would load.
  • Write the entire thing in C++ for speed and true multithreading and use Python solely for extensions. This would be a little more difficult since I kind of leverage the dynamic typing thing that Python has going on.

So far, my current structure has everything based on a Model which contains Blocks. In addition, there is a simulation Context which holds information about each Block and where the simulation is in terms of the current step (simulations are stepped over time (dt)). Contexts are also where a Block will store all of its information that it needs to retain during the current step and in the next step. A Block is an operation over time in the flow of the simulation: it could be a simple addition, maybe a derivative or integral, or it could be a full PID controller. Blocks declare themselves to have a number of Input objects and Output objects. Inputs/Outputs are named and have a slot/function called set which sets the value of the input or output. Outputs have a signal called ‘ready’ which inputs connect their ‘set’ slot to. When an output’s ‘set’ method is called, it emits its signal. When an input is set and it sees that all inputs attached to its block are set, it performs a “step” on the block. In addition, there are 3 special blocks: An EntryBlock, ExitBlock, and ModelBlock. Entry and Exit blocks are used in models since a model can have “Entry” and “Exit” points. These points can be used to loop a value from an Exit to an Entry (if they have the same name) or can be used as Input and Output objects if the Model is placed inside a ModelBlock. ModelBlocks are blocks which contain a model which they execute in a child simulation Context to their context. In this way, blocks can be nested. If one creates a Model with 2 Entries and 2 Exits with a pair of those Entries and Exits having the same name and then the Model is attached to a ModelBlock, the ModelBlock will have 1 input and one output to corrispond to the free Entry and Exit on the Model. Models can’t be recursive, but they can be nested so as long as a higher level block doesn’t contain a block which at some child level contains the same higher level block, there can be some sense of re-usability and modularity to a simulation.

Blocks are subclassed into a package called model. The __init__.py file in the model package defines the basic form for a block and then the individual modules in the package define more specific blocks. The blocks then have their constructors cached by reflection so that a block can be constructed by simply naming its name. To extend the blocks available in a simulation, all that must be done is to drop the new module python file into the model folder. I am considering changing this a bit to separate out user-added modules from the “system” modules in kind of the same fashion as I did with the WebSocketServer where I had the files in a folder be loaded into the context of another package.

Simulations are to be stored in an XML format which is going to be more or less human readable and should preserve the look and feel of the simulation. I am still working on the exact format at the moment, but that is the next step.

As for the GUI, I plan on using Qt since it seems the most cross-platform (sorry GTK…Windows needs too much help to load you and PyDev in eclipse doesn’t like the whole introspection thing). I plan on releasing the project under the Apache License (but don’t yet quote me on that or hold me to it…I may choose a different license later once I get more of a feel for how the project would be used). Either way, I plan on publishing the source code on github since it looks like nothing like this really exists in a simple form. Sure, there are clones of Simulink to work with Octave and things like that, but it doesn’t look like there are few, if any, stand-alone applications that do this (except perhaps a paid program called logic.ly, but this should be able to duplicate the functionality of that program as well). I guess it is kind of a niche market since the only people who do this kind of thing usually can afford Simulink and Matlab.

For the record, I do have access to Simulink and Matlab through the University I am attending, but where would the fun be in that?

The first week or two with Arch Linux

After some frustrating times involving Ubuntu 12.04, hibernation, suspending, and random freezing I decided I needed to try something different. Being a Sandy Bridge desktop, my computer naturally seems to have a slight problem with Linux support in general. Don’t get me wrong, I really like my computer and my processor…however, the hardware drivers at times frustrate me. So, at my wits end I decided to do something crazy and take the plunge to a bleeding edge rolling release linux: Arch Linux.

Arch Linux is interesting for me since its the first time I have not been using an operating system with the “version” paradigm. Since its a rolling release it is prone to more problems, but it also gives the advantage of always being up to date. Since my computer’s hardware is relatively new (it has been superseded by Ivy Bridge, but even so its driver support still seems to be being built), I felt that I had more to gain from doing a rolling release where new updates and such would come out (think kernel 3.0 to 3-2…Sandy Bridge processors suddenly got much better power management) almost immediately. So, without further adieu, here are my plusses and minuses (note that this will end up comparing Ubuntu to Arch alot since that’s all I know at the moment):

Plusses:

  • It was actually very easy to install. Since I have had problems with net installs, I did a core install and then updated it. I practiced several times beforehand on virtual machines, including using existing partitions and such. Although the initial downloads took some time (the lack of curl in the core install was kind of upsetting since I couldn’t use rankmirrors to get better speeds), after that it was pretty fast. Thanks to considerable documentation and a few practice runs, getting an X enviroment set up using Gnome 3 (and gdm…I like the graphical logins) didn’t take long at all. It took a bit of coaxing (read: google + arch wiki) to get things such as networkmanager running and such, but with time I had it all figured out and I managed to get the whole system running more or less stably within a day.
  • It boots faster than Ubuntu and is more explicit about what exactly it’s doing. I liked the Ubuntu moving logo thing, but I do actually enjoy seeing what the computer is doing when it boots. Coming from pure asthetic reasons, it gives the computer more of a “raw” feel which for some strange masochistic reason, I enjoy. The slowest part is initializing the networks and if that didn’t have to happen the entire system could boot in under 60 seconds after the bios gets done showing off its screen.
  • The documentation is awesome. Clearly, people have spent lots and lots and lots of time writing the documentation in the wiki for Arch. It certainly made setting up easier since many of the random corner cases were in the troubleshooting section of severl articles and I ended up running into a couple of them. One thing that was easier to set up than in Ubuntu was suspending and hibernating (at least getting it to work reliably). With some help from the forum (see next point) and a few pages of documentationon pm-utils I got suspend, resume, and hibernate (!!!) running. I haven’t even gotten hibernate to work in Windows.
  • The community is great. I rarely have been able to get a question answered on the Ubuntu forums since they are so conjested. I asked a question on the arch bbs and in less than a day I had a response and was able to do some trial + error and troubleshooting involving the suspend and hibernate functionality of my computer.

Minuses:

  • The rolling release model breaks things occasionally. Recently, the linux-firmware package was updated and this caused my wireless card to stop working since it could no longer find the drivers. I wasn’t sure why, but I have just downgraded the package and blacklisted it for upgrades. Hopefully that doesn’t kill me later (it probably will), but if it does by then I hope to have figured out what is wrong.
  • With great power comes great responsibility. The sheer flexibility is great since I don’t have a bunch of extra packages I don’t need, but at the same time when I was practicing with the VMs, I was able to get myself stuck in a hole where the only solution was to re-format the drive. However, ever since a mishap with Ubuntu (the themeing engine changed all my stuff to black on black or white on white for the text) I have separated out my home folder from the system so that I can easily re-format and re-install the system without losing all my stuff (all 132Gb of it).
  • This isn’t a problem for most people, but it doesn’t access the hard drive as often as other distros. Why is that a con for me? Well, I have a western digital green hard drive which has an automated parking feature which parks the heads after 10 seconds of inactivity. Well, in windows this doesn’t matter, but in linux since it touches the filesystem every 11-15 seconds or so, that results on a LOT of head parkings. Considering that the heads are only rated for 300K cycles and people have reported reaching that in less than a year, it is a real issue. I have a program (wdantiparkd) which writes the hard drive every 7 seconds while watching to see if anything else has written to the hard drive so that it hangs up after 10 minutes rather than 10 seconds. It helps, but it worked better on Ubuntu.

Overall, this experience with Arch has allowed me to become much more familiar with Linux and its guts and slowely but surely I am getting better at fixing issues. If you are considering a switch from your present operating system and already have experience with Linux (especially the command line since that’s what you are stuck with starting out before you install xfce, Gnome, KDE, etc), I would recommend this distribution. Of course, if you get easily frustrated with problems and don’t enjoy solving them, perhaps a little more stability would be something to look for instead.

Here is my desktop as it stands:

Arch Linux with Gnome 3

Arch Linux with Gnome 3

Case LEDs Software

So, I have just cleaned up, documented a little better, and zipped up the firmware and host side driver for the case LEDs. The file does not contain the hardware schematic because it has some parts in it that I created myself and I don’t feel like moving all the symbols around from my gEDA directory and getting all the paths to work correctly.

The host side driver only works on linux at the moment due to the usage of /proc/stat to get CPU usage, but eventually I plan on upgrading it to use SIGAR or something like that to support more platforms once I get a good environment for developing on Windows going. If you can’t wait for me to do it, you could always do it yourself as well.

Anyway, the file is here: LED CPU Monitor Software

Here is the original post detailing the hardware along with a video tour/tutorial/demonstration: The Case LEDs 2.0 Completed