Kevin Cuzner's Personal Blog

After some frustrating times involving Ubuntu 12.04, hibernation, suspending, and random freezing I decided I needed to try something different. Being a Sandy Bridge desktop, my computer naturally seems to have a slight problem with Linux support in general. Don't get me wrong, I really like my computer and my processor...however, the hardware drivers at times frustrate me. So, at my wits end I decided to do something crazy and take the plunge to a bleeding edge rolling release linux: Arch Linux.

Arch Linux is interesting for me since its the first time I have not been using an operating system with the "version" paradigm. Since its a rolling release it is prone to more problems, but it also gives the advantage of always being up to date. Since my computer's hardware is relatively new (it has been superseded by Ivy Bridge, but even so its driver support still seems to be being built), I felt that I had more to gain from doing a rolling release where new updates and such would come out (think kernel 3.0 to 3-2...Sandy Bridge processors suddenly got much better power management) almost immediately. So, without further adieu, here are my plusses and minuses (note that this will end up comparing Ubuntu to Arch alot since that's all I know at the moment):

Plusses:

• It was actually very easy to install. Since I have had problems with net installs, I did a core install and then updated it. I practiced several times beforehand on virtual machines, including using existing partitions and such. Although the initial downloads took some time (the lack of curl in the core install was kind of upsetting since I couldn't use rankmirrors to get better speeds), after that it was pretty fast. Thanks to considerable documentation and a few practice runs, getting an X enviroment set up using Gnome 3 (and gdm...I like the graphical logins) didn't take long at all. It took a bit of coaxing (read: google + arch wiki) to get things such as networkmanager running and such, but with time I had it all figured out and I managed to get the whole system running more or less stably within a day.
• It boots faster than Ubuntu and is more explicit about what exactly it's doing. I liked the Ubuntu moving logo thing, but I do actually enjoy seeing what the computer is doing when it boots. Coming from pure asthetic reasons, it gives the computer more of a "raw" feel which for some strange masochistic reason, I enjoy. The slowest part is initializing the networks and if that didn't have to happen the entire system could boot in under 60 seconds after the bios gets done showing off its screen.
• The documentation is awesome. Clearly, people have spent lots and lots and lots of time writing the documentation in the wiki for Arch. It certainly made setting up easier since many of the random corner cases were in the troubleshooting section of severl articles and I ended up running into a couple of them. One thing that was easier to set up than in Ubuntu was suspending and hibernating (at least getting it to work reliably). With some help from the forum (see next point) and a few pages of documentationon pm-utils I got suspend, resume, and hibernate (!!!) running. I haven't even gotten hibernate to work in Windows.
• The community is great. I rarely have been able to get a question answered on the Ubuntu forums since they are so conjested. I asked a question on the arch bbs and in less than a day I had a response and was able to do some trial + error and troubleshooting involving the suspend and hibernate functionality of my computer.

Minuses:

• The rolling release model breaks things occasionally. Recently, the linux-firmware package was updated and this caused my wireless card to stop working since it could no longer find the drivers. I wasn't sure why, but I have just downgraded the package and blacklisted it for upgrades. Hopefully that doesn't kill me later (it probably will), but if it does by then I hope to have figured out what is wrong.
• With great power comes great responsibility. The sheer flexibility is great since I don't have a bunch of extra packages I don't need, but at the same time when I was practicing with the VMs, I was able to get myself stuck in a hole where the only solution was to re-format the drive. However, ever since a mishap with Ubuntu (the themeing engine changed all my stuff to black on black or white on white for the text) I have separated out my home folder from the system so that I can easily re-format and re-install the system without losing all my stuff (all 132Gb of it).
• This isn't a problem for most people, but it doesn't access the hard drive as often as other distros. Why is that a con for me? Well, I have a western digital green hard drive which has an automated parking feature which parks the heads after 10 seconds of inactivity. Well, in windows this doesn't matter, but in linux since it touches the filesystem every 11-15 seconds or so, that results on a LOT of head parkings. Considering that the heads are only rated for 300K cycles and people have reported reaching that in less than a year, it is a real issue. I have a program (wdantiparkd) which writes the hard drive every 7 seconds while watching to see if anything else has written to the hard drive so that it hangs up after 10 minutes rather than 10 seconds. It helps, but it worked better on Ubuntu.

Overall, this experience with Arch has allowed me to become much more familiar with Linux and its guts and slowely but surely I am getting better at fixing issues. If you are considering a switch from your present operating system and already have experience with Linux (especially the command line since that's what you are stuck with starting out before you install xfce, Gnome, KDE, etc), I would recommend this distribution. Of course, if you get easily frustrated with problems and don't enjoy solving them, perhaps a little more stability would be something to look for instead.

Here is my desktop as it stands:

I have finally given in and purchased some hosting at rackspace so I could put the websocketserver somewhere. The demo is live and working, so just visit the following URL to try a very super simple chatroom:

http://cuznersoft.com/python-websocket-server/html/demo_chatroom.html

After spending way to much time thinking about exactly how to do it, I have got multiprocessing working with the WebSocketServer.

I have learned some interesting things about multiprocessing with Python:

• Basically no complicated objects can be sent over queues.
• Since pickle is used to send the objects, the object must also be serializable
• Methods (i.e. callbacks) cannot be sent either
• Processes can't have new queues added after the initial creation since they aren't picklable, so when the process is created, it has to be given its queues and that's all the queues its ever going to get

So the new model for operation is as follows:

Sadly, because of the lack of ability to share methods between processes, there is a lot of polling going on here. A lot. The service has to poll its queues to see if any clients or packets have come in, the server has to poll all the client socket queues to see if anything came in from them and all the service queues to see if the services want to send anything to the clients, etc. I guess it was a sacrifice that had to be made to be able to use multiprocessing. The advantage gained now is that services have access to their own interpreter and so they aren't all forced to share the same CPU. I would imagine this would improve performance with more intensive servers, but the bottleneck will still be with the actual sending and receiving to the clients since every client on the server still has to share the same thread.

So now, the plan to proceed is as follows:

• Figure out a way to do a pre-forked server so that client polling can be done by more than one process
• Extend the built in classes a bit to simplify service creation and to make it a bit more intuitive.

As always, the source is available at https://github.com/kcuzner/python-websocket-server

For the past few weeks I have been experimenting a bit with HTML5 WebSockets. I don't normally focus only on software when building something, but this has been an interesting project and has allowed me to learn a lot more about the nitty gritty of sockets and such. I have created a github repository for it (it's my first time using git and I'm loving it) which is here: https://github.com/kcuzner/python-websocket-server

The server I have runs on a port which is considered a dedicated port for WebSocket-based services. The server is written in python and defines a few base classes for implementing a service. The basic structure is as follows:

Each service has its own thread and inherits from a base class which is a thread plus a queue for accepting new clients. The clients are a socket object returned by socket.accept which are wrapped in a class that allows for communication to the socket via queues. The actual communication to sockets is managed by a separate thread that handles all the encoding and decoding to websocket frames. Since adding a client doesn't produce much overhead, this structure potentially could be expanded very easily to handle many many clients.

A few things I plan on adding to the server eventually are:

• Using processes instead of threads for the services. Due to the global interpreter lock, if this is run using CPython (which is what most people use as far as I know and also what comes installed by default on many systems) all the threads will be locked to use the same CPU since the python interpreter can't be used by more than one thread at once (however, it can for processes). The difficult part of all this is that it is hard to pass actual objects between processes and I have to do some serious re-structuring for the code to work without needing to pass objects (such as sockets) to services.
• Creating a better structure for web services (currently only two base classes are really available) including a generalized database binding that is thread safe so that a service could potentially split into many threads while not overwhelming the database connection.

Currently, the repository includes a demo chatroom service for which I should have the client side application done soon and uploaded. Currently it supports multiple chatrooms and multiple users, but there is no authentication really and there are a few features I would like to add (such as being able to see who is in the chatroom).

So, I have just cleaned up, documented a little better, and zipped up the firmware and host side driver for the case LEDs. The file does not contain the hardware schematic because it has some parts in it that I created myself and I don't feel like moving all the symbols around from my gEDA directory and getting all the paths to work correctly.

The host side driver only works on linux at the moment due to the usage of /proc/stat to get CPU usage, but eventually I plan on upgrading it to use SIGAR or something like that to support more platforms once I get a good environment for developing on Windows going. If you can't wait for me to do it, you could always do it yourself as well.

Anyway, the file is here: LED CPU Monitor Software

Here is the original post detailing the hardware along with a video tour/tutorial/demonstration: The Case LEDs 2.0 Completed

After much pain and work...(ok, I had a great time; let's be honest now)...I have finished the case LEDs!

Pursuant to the V-USB licence, I am releasing my hardware schematics and the software (which can be found here). However, it isn't because of the licence that I feel like releasing them...it is because it was quite fun to build and I would recommend it to anyone with a lot of time on their hands. So, to start off let us list the parts:

• 1 ATMega48A (Digi-key: ATMEGA48A-PU-ND)
• 1 28 pin socket (Digi-key: 3M5480-ND)
• 2 3.6V Zener diodes (Digi-key: 568-5907-1-ND)
• 2 47Ω resistors (Digi-key: 47QBK-ND)
• 1 39Ω resistor (Digi-key: 39QTR-ND)
• 1 15Ω resistor (Digi-key: 15H-ND)
• 3 100V 300mA TO-92 P-Channel MOSFETs (Digi-key: ZVP2110A-ND)
• 3 2N7000 TO-92 N-Channel MOSFETs (Digi-key: 2N7000TACT-ND)
• 1 10 Position 2x5, 0.1" pitch connector housing (Digi-key: WM2522-ND)
• 10 Female terminals for said housing (Digi-key: WM2510CT-ND)
• 1 4-pin male header, 0.1" pitch for the diskette connector from your power supply (You can find these on digikey pretty easily as well..there are a lot)
• 2 RGB LEDs (Digi-key: CLVBA-FKA-CAEDH8BBB7A363CT-ND, but you can you whatever you may find)
• 4 White LEDs like in my last case mod
• 1 Prototyping board, 24x17 holes

The schematic is as follows:

Schematic (click to open full size)

The parts designations are as follows:

• R1: 15Ω
• R2: 39Ω for the Red channel
• R3: 47Ω for the Green channel
• R4: 47Ω for the Blue channel
• LED1-4: White LEDs of your choosing. Make sure to re-calculate the correct value for R1, taking into account that there are 4 LEDs
• LED5-6: The RGB LEDs. The resistor values here are based on the part I listed above, so if you decide to change it, re-calculate these values.
• Q1-Q3: The P-Channel MOSFETs
• Q4-Q6: The N-Channel MOSFETs
• Z1-Z2: The zener diodes
• U1: 16Mhz Crystal
• C1-2: Capacitors to match the crystal. In my circuit, I think they were 33pF or something
• CONN-PWR: The 4-pin connector for the diskette
• CONN-USB: The USB connector. You will have to figure out the wiring for this for your own computer. I used this site for mine. Don't forget to twist the DATA+ and DATA- wires if you aren't using a real USB cable (like me).
• C3: Very important decoupling capacitor. Place this close to the microcontroller.

As I was building this I did run into a few issues which are easy to solve, but took me some time:

• If the USB doesn't connect, check the connections, check to make sure the pullups are in the right spot, and check to make sure the DECOUPLING CAPACITOR is there. I got stuck on the decoupling capacitor part, added it, and voila! It connected.
• If the LEDs don't light up, check the connections, then make sure you have it connected to the right power rails. My schematic is a low-side switch since the LEDs I got were common anode. I connected both ends to negative when I first assembled the board and it caused me quite a headache before I realized what I had done
• Double and triple check all the wiring when soldering. It is pain to re-route connections (trust me...I know). Measure twice, cut once.

Although I already have a link above, the software can be found here: Case LEDs Software

So, here are pictures of the finished product:

LEDs shining magenta

LEDs shining orange

LEDs shining green

With its guts hanging out

The mounting viewed from the outside

Mounted onto the front fan grille

As I was working to get my led case lights ready to install (tutorial and videos coming soon!) I was trying to make the device interchangeable between windows and linux since I run both on my computer. So, I followed some ...great... online instructions which directed me to install something called ReadyDriver Plus. Basically, what it does is disable device driver signing so that you can install unsigned drivers such as libusb. Well, the first time I restarted, it did something weird. You see, I have a setup where my computer first loads GRUB and then from GRUB it will load the windows bootloader. I think that was the main problem with getting ReadyDriver Plus to work since all it is is a program that loads on boot to disable device signing in the startup menu for windows. However, because it didn't work properly it had the added effect of disabling ALL of my device drivers, including my wired keyboard and bluetooth dongle for the mouse. Great. So, I then manually told it to disable device driver signing (F8 when selecting Windows 7 from the windows bootloader, scroll down to "Disable driver signing") and it loaded again just fine. Then, I said to myself, "Oh, I can just do it that way when I want to use the lights in my computer on windows" and so I decided to un-install ReadyDriver Plus. Bad idea. Now, it doesn't show the bootloader screen for the windows bootloader and I discovered that the keyboard driver is no longer working again. So, I can't tell it to turn off device signing and I can't use the computer in windows mode. Great. Now, it seems the easiest thing to do is re-install windows. Hopefully it doesn't mind since it is on the same computer (read: motherboard) due to their crazy DRM stuff they put in windows 7.

In short, I don't think windows will be supported for my case lights just yet.

EDIT: Yay for system restore! Apparently when I installed Assassins Creed it created a nice system restore point. Now everything works great again. However, I still won't be messing with the drivers again for a while. I have to make sure I have plenty of time on my hands before I do that again just in case system restore isn't as awesome.

So, as usual after I completed my LED case mod I asked myself, how can I could make it even cooler? Thus was born the idea for Case LEDs v. 2.0.

The Idea:  Wire up some LEDs so they are controlled by the computer to vary their intensity or something based on the CPU usage.

The Implementation:  Using RGB LEDs, some small MOSFETs, and a microcontroller make a USB controlled light generator that takes as input a number representing CPU usage.

In the 3 weeks since I put the white LEDs in my case I have been working on this thing in my spare time (mostly weekends...homework has just been swamping me during the week) and this past weekend I finally got it to connect through the USB using the V-USB library and so I have made a lot of progress. At the moment it is perfectly capable of displaying CPU usage by way of color (it is really cool to watch), but I still want to add a few features before I release the source code (and I also need to test it to make sure it doesn't crash after 2 days or have some horrible memory leak in the host software or something...).

Since I am running linux, the host software was developed linux specific, but later I will add support for Windows since I plan on installing Windows on my computer for gaming at some point. There are two parts to the software: The device firmware and the host software. To minimize USB traffic, the firmware does the conversions from cpu usage to RGB and also the visual efffects. All the host software has to do is read the cpu usage and tell the device about it.

The hardware isn't incredibly complex: It uses an ATMega168A microcontroller (I am going to be aiming for a smaller 14-18 pin microcontroller eventually...this one is just too big and it would be a waste) to control some MOSFETs that turn on and off the LEDs. The LEDs I got were some $0.55 4-PLCC ones from Digikey which I have soldered some wires to and secured with hot glue (my first try looks awful with the hot glue everywhere...the 2nd one looks amazing since I figured out that hot glue melts before heat shrink shrinks so I could put the glue inside the heat shrink). There are 2 MOSFETs per LED channel in a complementary logic configuration. Since the LEDs are common anode, the MOSFETs control the cathode wire and so there isn't an inverting effect (put in a 1, get out a 0 and vice versa) like what usually happens with complementary logic. The whole system runs at 3.3V since I didn't have any 3.6V zener diodes to use for the USB pins to keep the voltage levels in check so that it would be able to talk to the computer. Apparently the voltage levels are very strict for USB and my first few tries of getting this to work didn't communicate with the computer because of the voltage levels coming out of the USB pins. After I changed the voltage to 3.3V it worked perfectly on the first try. Eventually this is going to connect to one of the internal USB connectors on the motherboard with power supplied by the same 4 pin connector I used for my white LEDs. I am debating running it entirely off USB power, but I am still not sure since that would limit any future expansion to 500mA of current draw and with the planned configuration it will be drawing between 250-300mA already.

Anyway...I plan on making a tutorial video of sorts along with pictures and schematics since in reality aside from the programming this was an easy project. I just need a week or two to get all the parts soldered together and the program finalized and then I'll know exactly how much this thing costs to build.

The computer

In November I purchased the parts for a new computer since mine was getting very old (I got it in 2006 and even then it wasn't exactly top of the line). I put it together and it has been performing admirably for a couple months now. I was researching graphics cards and it occurred to me that I would have to move my hard drive up a slot to fit a large graphics card in my case.

After moving stuff around inside

So, I opened the case and started moving stuff around. I also decided to re-organize the cables so that they wouldn't be dangling precariously above the CPU fan by stuffing them behind the HDD cage. During that process I took some strain off the SATA cables (they are kind of stiff and I think they were putting undue stress on the sockets on the motherboard, so I moved them around so that they wouldn't be so convoluted). After finishing all this it occurred to me that my case would look sweet if I were to add some LEDs to it. I then set out to install some LEDs.

The grille and power connector

In the front of the case there is a plastic piece that covers the metal body of the case and also holds the power button, reset button, and HDD light. This case has a grille on it to allow air to pass through into the front fan (if I had one installed).

I decided that this grille could look awesome if it had some backlighting. I had considered using a lighted fan for this purpose before, but since fans are mounted on the inside of the case it would project the shadows from the internal metal structure onto the plastic grille, ruining the effect. I decided to mount some white LEDs on the inside of the plastic piece pointing towards the inside of the case so they could shine on and illuminate the part behind the grille to give a "glowing" effect. Here is what I used:

• Some spare really thick black matte cardstock my sister let me have (she is into artsy things)
• 4 White LEDs that I had lying around
• A 15Ω resistor to limit the current (4 LEDs @ 25mA each comes to 100mA at a voltage drop of 3.5V)
• A .1" header I had in a parts box
• Some wire
• Some tape

The spider wires

I started out by soldering the header to some wires to take the 5V and GND line off of the small .1" power connector in my computer. I then put the resistor on the positive rail and then split everything off into 4 wires (8 in total: 4 power, 4 ground). The result looked rather like a spider in my opinion. After that it was a relatively simple job of soldering the long lead of the LEDs to the positive rail and the other side to the negative rail. Thus, the LED assembly was completed.

Matte board and aimed LEDs

The more difficult part was attaching the matte board to the metal part of the case and then aiming the LEDs. The matte board was necessary because without it the LEDs reflected a little too well off the metal of the case and they could be clearly seen through the grille. I cut the matte board into two pieces large enough to cover the metal on either side of the grille and used tape to hold it in place. One hitch came up with the wires going to the front of the case: the hole for the wires was right beneath one of the grilles and was not easily covered by the cardstock. I ended up just basically laying the cardstock over the hole and wires and moving them around so as to not be visible through the grille. The next bit of matte board I used was to create a shroud of sorts around the HDD and power lights since the LEDs were bright enough that they shined through the bezels for those lights as well. I then spent a while aiming the lights until I was satisfied and then I put the computer back together so I could enjoy my new lights.

The Final Effect

All in all my specs are as follows:

• ASRock Z68 Pro3-M Motherboard (microATX)
• Intel Core i5-2500K Processor running at the stock frequency
• 8GB G.Skill Ripjaws DDR3 RAM
• 1TB Western Digital Green HD (using wdantiparkd to counter the head parking issues)
• Rosewill RNX-N300X Wireless b/g/n card
• Apex 500W Power Supply
• Cheap DVD burner
• Cheap Rosewill card reader
• Apex microATX case with a handle (best feature!) and some LEDs
• Ubuntu 11.10 64-bit for an OS

Since I never quite finished the story about my dot matrix clock, I see no reason why I shouldn't write a bit of a continuation of my current developments. Shortly before I left on my mission for my church in November 2009, I received my boards for the dot matrix clock and assembled them. However, I ran into a problem: The displays would turn off after the voltages on the gates of the row driving mosfets reached a certain voltage and when the voltage was at a level where it would turn on the display, it had some problems with turning on and off the LEDs. Now, after I left on my mission I would think about this once in a while and I figured out the problem: I was using N-Channel mosfets with only 5V or less of gate driving voltage so they wouldn't turn on/off all the way. I am sure there are more problems than just that, but I keep kicking myself for using N-Channel mosfets instead of using P-Channel. Had I used P-Channel, the problem would have been avoided and this whole thing would have worked great. For the moment, however, this project is on hold since I am designing and building a few things that I will need in the long term here at college since I can't lug around a power supply and an oscilloscope.

So, in summary, if I were to do a re-design I would change the following:

• The row drivers would be P-Channel mosfets. This would require using something other than a 4->16 demux for a gate driver unless I could find one with inverted outputs. Even with that I would probably put some very small gate drivers (if they exist...the size restriction might be too much) as a buffer to ensure the mosfets were turning on and off properly.
• I would factor in larger tolerances. If there was one design lesson I learned from getting these boards it is that I need to make sure that I make the holes for things a little larger. It would definitely make assembly easier.
• The PIC18F4550 would be replaced for a ATMega of some sort or maybe even a small FPGA. I was running into speed problems with the 18F4550 with getting refresh rates up (I know this is contrary to previous posts, but I was starting to have problems getting 30-60fps like I wanted...even though the image wasn't showing up anyway because of the mosfets). The 12MIPS speed was just a tad too slow and so I think if I were to use a 20MIPS ATMega it would work a lot better. Also, the tools for ATMega seem to be a little more opensourced than the ones for the PIC. I say this because avrgcc runs much better on my Linux machines than the various C compilers for the PIC series. Also, my AVR programmer (a USBASP) has very good native Linux support.

Now all that is left for me to do is to figure out how I can modify the boards I have so that I wouldn't have to drop $70 again...