Article

Unknown date

24 July 2012, 06:07 UTCLinux 3.5 on the GTA04

Linus released Linux-3.5 a couple of days ago so it is long past time to have 3.5 available for the OpenPhoenux GTA04. By the time I publish this blog entry it will be, both on github at https://github.com/neilbrown/linux/tree/gta04/3.5.y and on my own server at http://neil.brown.name/git?p=gta04;a=shortlog;h=refs/heads/3.5-gta04.

Quite a lot has gone into 3.5-gta04. A lot of that was bug fixes. 3.5 seems to have more changes that break my phone than other recent kernels, though I haven't counted bugs or hours so that could be misleading. Certainly I have quite a lot of patches in my 'bugs' branch which is mainly for fixing regressions, but then 8 of those are all related: the libertas has a new async firmware loader which seems to be broken. I haven't examined it closely, but wifi didn't work until I revert those patches.

On a happier note I've found time to make various improvements.

I think that is all. Power management seems to be largely unchanged - 25mA when in suspend, about 70mA when idle, awake, screen off, and about 200mA when idle and awake with screen on. I'll be using this on my phone on a regular basis and see how it goes.

[permalink (No comments)]


08 July 2012, 10:31 UTCOpenPhoenux Serial Console Access

I find that serial console access is a must for debugging kernel problems on the GTA04 board in my OpenPhoenux phone. Getting serial console access to the board itself is quite easy as it has a little connector on the board, and came with a ribbon cable which plugs in to this connector and has a DB-9 on the other end.

However it isn't so easy when the board is in the phone case, which is where it is most useful. The connector is on the side of the board closest to the front of the phone, just above the display. Cutting a hole there might be possible, but having a ribbon cable hanging out isn't really an option. So I needed to be a little more clever.

Firstly I destroyed the ribbon cable, keeping the brown, yellow, and blue wires and cutting the rest off at the plug. The three wires I kept about 2cm of length on. You can probably see some solder burns from an earlier experiment where I tried to solder 3 wires directly onto the pins at the back of the plug. That worked a little bit, but they kept breaking off. Maybe I'm not good a soldering, or maybe it was just a bad idea.

As you can see in this picture, the wires are feed around the edge of the board the the back. They are aiming for the little cavity beside the battery.

Then I cut away some plastic to make a hole big enough for a small cable with tap wrapped around it, shown here.

Next I found a nice small 3.5mm stereo jack of an old busted pair of headphone. Not all jacks are small enough. Ipod headsets seem to have particularly slim jacks, but others probably work as well. Soldering the 3 wires from this to the 3 wires from the ribbon cable means that I have TX, RX, GND in a reasonably accessible location.

You need to be careful about getting the length of the cable right so it is long enough without any excess. Here you see it tucked away so the back cover can go over. The tip of the jack goes into the slot a little bit and as you might be able to see, I have a little bit too much cable so it buckles a bit an uses more space than I would like. I can clip the back on though, which is all that matters.

Finally I bought a short cable with a female 3.5mm on one end (I think there were 2 RCA on the other), cut that up, and soldered the 3 wires to my DB-9. And now I get console access by just popping the cover over (being careful not to let the battery fall out), easing the jack out, and plugging it in. Very useful.



[permalink (No comments)]


06 July 2012, 12:29 UTCTedious bugs

Some bugs can be educational, but others are just tedious. I guess one still learns, but there must be a better way.

One of the many issues with my Open Phoenux GTA04 is that it hangs occasionally. Note that I'm not complaining about there being issues. I deliberately got this phone because I wanted to experiment and explore and if it all worked perfectly, where would the fun be? But there are issues and some are fun to fix. Others..

So anyway, it hangs. The symptom is that sometimes I ask it to wake up from suspend and it doesn't. The console is completely silent (no sysrq even). This seemed to correlate with someone trying to call or TXT me, and the message not getting through. So my initial assumption was that it would hang on resume. But only occasionally.

So to find out more I installed a kernel with debugging enabled (this is a 3.2 based kernel, it currently seems most stable), connected a serial console, (via a 3.5mm phono jack that I manage to wire up) and programmed the phone to set the RTC alarm every minute (fortunately I got the RTC alarm working!) so it would suspend and resume every 60 seconds. Then let it go.

Within about 30 minutes it would be very likely to hang.... providing the USB cable wasn't plugged in. Even after I discovered this requirement I kept leaving the USB cable in after copying a new kernel across, and so wasted lots of testing time - I would leave it for 30 minutes with the cable in, and then realise that it wasn't going to hang like that. Like I said: tedious.

The first hang showed that it was just after "PM: early resume of devices complete" so I added some more tracing and found that it was in resume_device_irqs(). Then more and it was just as interrupt 92 - musb-hdrc - was enabled. There was a pending interrupt for this device and something got confused in there. A few more cycles (about an hour each!) found the bug for me.

omap2430_musb_set_vbus in omap2430.c contains:

while (musb_readb(musb->mregs, MUSB_DEVCTL) & 0x80) { cpu_relax(); if (time_after(jiffies, timeout)) { dev_err(musb->controller, "configured as A device timeout"); ret = -EINVAL; break; } }

where 'timeout' is set one second in the future. Now looping for up to one second in code that can be called from an interrupt handler (as this is called from musb_stage0_irq), is pretty poor form to start with, but it is worse than that.

When resume_device_irqs() calls __enable_irq() which checks for pending interrupts and calls the handler, it does this with all interrupts disabled (and having interrupts disabled in an interrupt handler is pretty common, so this should not be a problem). With interrupts disable, the timer doesn't tick, so jiffies doesn't ever change. You can see where this is going, can't you.

If that interrupt is pending on resume, we get to this code and don't just spin for 1 second, we spin forever - with interrupts disabled. This exactly matches what I saw.

Working around this should be easy - just put in a maxiumum loop count ... I'll get to that in a minute. But why is there an interrupt at all?

Well the interrupt handler - generic_interrupt() in musb_core.c - reads the interrupt status registers and the only bit that is set is MUSB_INTR_SESSRQ in int_usb. i.e. There is a request to start a new session.

I *think* this means that the ID pin in the USB port was grounded. A USB-OTG cable will do this to request that the USB-OTG port switch to host mode and start talking to a gadget. This is what the code seems to be doing. In the first instance it turns on VBUS (the 5V power supply for the USB). So it all seems consistent.

However I never plugged anything into the USB port, and definitely didn't ground the ID pin. Remember that when I did have the USB port plugged in the problem didn't happen. Maybe that is related.

My guess is that some electrical noise either during suspend or resume triggers the interrupt. Maybe there really should be a capacitor on that 'ID' pin - it wouldn't be the first time that a missing capacitor had caused problems for Openmoko devices. I don't know how to test this, or whether to care if I can just make it work.

But it seems I cannot, at least not yet.

I imposed a 1000-loops count on that loop and it got further, but not much. It took about 5 second (so I can make the max a lot smaller), the interrupt handler gave up, and we moved on to resume all devices (having completed early-resume). It got up to the MMC drivers (hsmmc), stopped for 180 seconds, then the soft-lockup timer triggered. So not it is not hanging with interrupts disabled, so it might be easier to debug, but it is still hanging.

The stack trace shows

[ 4758.457092] mmcqd/0 D c0449efc 5644 59 2 0x00000000 [ 4758.463806] [<c0449efc>] (__schedule+0x57c/0x608) from [<c044a3e0>] (schedule_timeout+0x1c/0x1d0) [ 4758.473052] [<c044a3e0>] (schedule_timeout+0x1c/0x1d0) from [<c044985c>] (wait_for_common+0xd8/0x150) [ 4758.482666] [<c044985c>] (wait_for_common+0xd8/0x150) from [<c02f4408>] (mmc_wait_for_req_done+0x24/0xa4) [ 4758.492645] [<c02f4408>] (mmc_wait_for_req_done+0x24/0xa4) from [<c02f4bc0>] (mmc_start_req+0x50/0x144) [ 4758.502410] [<c02f4bc0>] (mmc_start_req+0x50/0x144) from [<c02fe460>] (mmc_blk_issue_rw_rq+0x78/0x4dc) [ 4758.512115] [<c02fe460>] (mmc_blk_issue_rw_rq+0x78/0x4dc) from [<c02fecc8>] (mmc_blk_issue_rq+0x404/0x434) [ 4758.522155] [<c02fecc8>] (mmc_blk_issue_rq+0x404/0x434) from [<c02ff824>] (mmc_queue_thread+0x98/0x100) [ 4758.531951] [<c02ff824>] (mmc_queue_thread+0x98/0x100) from [<c0055060>] (kthread+0x80/0x88) [ 4758.540771] [<c0055060>] (kthread+0x80/0x88) from [<c000f30c>] (kernel_thread_exit+0x0/0x8)

which suggests that a block io request to the mmc is being retried immediately on resume, and not getting anywhere. Now it could be that this is completely unrelated to the USB and I should run more tests, but not today.

The suspend/resume thread is :

[ 4758.669219] susman D c0449efc 5060 1423 1420 0x00000000 [ 4758.675872] [<c0449efc>] (__schedule+0x57c/0x608) from [<c02f36c8>] (__mmc_claim_host+0xb8/0x154) [ 4758.685119] [<c02f36c8>] (__mmc_claim_host+0xb8/0x154) from [<c02f8edc>] (mmc_sd_resume+0x34/0x5c) [ 4758.694458] [<c02f8edc>] (mmc_sd_resume+0x34/0x5c) from [<c02f2b20>] (mmc_resume_host+0xc8/0x15c) [ 4758.703704] [<c02f2b20>] (mmc_resume_host+0xc8/0x15c) from [<c0307874>] (omap_hsmmc_resume+0xa0/0xe4) [ 4758.713317] [<c0307874>] (omap_hsmmc_resume+0xa0/0xe4) from [<c024e760>] (platform_pm_resume+0x44/0x54) [ 4758.723114] [<c024e760>] (platform_pm_resume+0x44/0x54) from [<c0252bcc>] (pm_op+0x6c/0xb8) [ 4758.731811] [<c0252bcc>] (pm_op+0x6c/0xb8) from [<c0253778>] (device_resume+0x190/0x228) [ 4758.740234] [<c0253778>] (device_resume+0x190/0x228) from [<c025391c>] (dpm_resume+0x10c/0x244) [ 4758.749298] [<c025391c>] (dpm_resume+0x10c/0x244) from [<c0253a60>] (dpm_resume_end+0xc/0x18) [ 4758.758178] [<c0253a60>] (dpm_resume_end+0xc/0x18) from [<c00768cc>] (suspend_devices_and_enter+0x1d0/0x22c) [ 4758.768432] [<c00768cc>] (suspend_devices_and_enter+0x1d0/0x22c) from [<c0076a54>] (enter_state+0x12c/0x18c) [ 4758.778656] [<c0076a54>] (enter_state+0x12c/0x18c) from [<c0075884>] (state_store+0x94/0x118) [ 4758.787536] [<c0075884>] (state_store+0x94/0x118) from [<c01d4d8c>] (kobj_attr_store+0x1c/0x24) [ 4758.796600] [<c01d4d8c>] (kobj_attr_store+0x1c/0x24) from [<c01172e4>] (sysfs_write_file+0x108/0x13c) [ 4758.806213] [<c01172e4>] (sysfs_write_file+0x108/0x13c) from [<c00bfc5c>] (vfs_write+0xac/0x180) [ 4758.815368] [<c00bfc5c>] (vfs_write+0xac/0x180) from [<c00bfde8>] (sys_write+0x40/0x6c) [ 4758.823699] [<c00bfde8>] (sys_write+0x40/0x6c) from [<c000e9c0>] (ret_fast_syscall+0x0/0x3c)

So it looks like it is waiting for the mmc device, which itself is hanging. My feeling here is that this is a very different problem. The block queue should block everything on suspend so no requests should be pending at this point. I guess I'm going to have to look into that some more.

So I've probably learned a bit, but really not much. Mostly just some very silly code in an interrupt handler, which took hours to find. Hopefully examining the block-device issue will be more fun.


UPDATE:

The MMC bug was fairly easy to find - the 'suspend' callback was simply not being called. This was easily fixed by applying commit 32d317c60e56c2a34463b51fc0336cc96b3e1735 from Linux-3.4.

So now my GTA04 doesn't crash on resume any more. Hurray.


[permalink (1 comment)]


24 February 2009, 19:53 UTCMeasuring Freerunner battery life [UPDATED]

I'm trying to avoid getting too distracted by my Openmoko Freerunner this week as I have a very different project that I want to concentrate on. But it is hard not to play with it sometimes, as it is just sitting there waiting. But fortunately I managed to find something useful to do with it that required me leaving it alone.

I thought it was time to find out how long the battery really lasts. So I fully charged it a couple of days ago and then left it sleeping with the intention of waking it up the next morning and seeing how much battery was left after a given time. Unfortunately I left it too long and the battery went completely flat, so I learned very little.

Now obviously I don't want to be checking it every so-often as that would distract me from my other project, and turning the display on all the time would be more power usage that I was wanting to measure. So I came up with the incredibly creative idea of getting the device to observe itself. That is of course the beauty of computers. They can do the work for you.

So here is the script that I wrote and ran, saving the output to a file:

while : do echo =========================================== date cat /sys/class/power_supply/battery/capacity cat /sys/class/i2c-adapter/i2c-0/0-0073/resume_reason cat /sys/class/i2c-adapter/i2c-0/0-0073/neo1973-resume.0/resume_reason /root/wkalrm +30m sleep 20 apm -s done

It very simply wakes up every 30 minutes, check the battery capacity and some other random bits of information that I wanted to check, stays away for 20 seconds, then goes back to sleep.

There are two reasons for to 20 second sleep. One is that I wanted to be able to ssh in and kill the script if I needed to get control of the device again before the power ran out completely. The other is that the Xglamo X server seemed to get confused if I suspend too soon after waking up. I guess it needs a little while to sort itself out after a resume. I haven't experimented with this much so I'm I may be misinterpreting a single failure with the wrong general cause.

And the results? The device just sat there for about 15 hours. The screen stays off the whole time. I might sometimes hear a little click from the speaker when it resumes, but that is the only external indication that anything is happening.

The sequence of battery capacity readings was

97 94 91 88 85 82 79 77 74 71 68 65 62 59 56 53 50 47 44 41 38 35 32 29 25 22 19 16 13 10

A very consistent difference of 3% every 30 minutes, except 79-77 where the difference is 2, and 29-25 where it is 4. So after 15 hours, 90% is gone. I was probably hoping for a bit more than that, but it should be workable.

This was with both the GSM and the GPS devices powered the whole time. And of course the CPU powered for 20 seconds every 30 minutes.

Later today after the device is fully charged, I'll try again with the GPS turned off. It might also be interesting to try with GSM off and GPS on. I assume Wifi and Bluetooth are turned of by suspend... I guess I should check that.

Update

So I tried with GPS and Bluetooth turned off. It didn't quite go as planned though. The samples I got are =========================================== Wed Feb 25 12:27:36 EST 2009 94 =========================================== Wed Feb 25 12:57:38 EST 2009 93 =========================================== Thu Feb 26 05:46:05 EST 2009 62

i.e. the first wakeup worked, but then it didn't wake again until I woke it to check the results. My guess is that my "go to sleep when idle" program put the device to sleep during the "sleep 20". Then when the alarm woke it, the script sent the device back to sleep never to awake. I'll try again with the auto-sleep disabled.

But the results are still good. Nearly 17 hours and only 32 percent gone. That is 1/3 of the power usage for then GPS and possibly BT were powered. So while keeping the GPS on for regular sampling is possible, it does eat battery life. The chip apparently has a power-save mode where it remembers all the state data but doesn't listen to the voices from the sky. I wonder if it is possible to enter that mode during suspend...

later ... A proper run with bluetooth and gps turned off gives these samples, one per half hour:

95 94 93 92 92 91 90 89 88 87 86 85 84 83 82 81 80 79 78 77 76 75 74 73 73 71 70 69 68 67

Which suggests 50 hours from full to empty. So charging once a day with light use should be fine.

The GPS does have a mode where it can sleep for a set period, wake up to get a new fix, then go back to sleep. I'll give that a try when I find time. I wonder what a good 'set period' is. 30 minutes? Maybe it depends on how much you are traveling.

But more immediately, I'll test with GPS off but Bluetooth on. I'm hoping it gets turned off at suspend, but these is a simpe way to test...

Final Update... The setting for bluetooth power made no difference. It seems it is turned off when we go into suspend, at least by default.



[permalink (6 comments)]


15 February 2009, 22:42 UTCtapinput: Yet another soft keyboard for the freerunner.

In theory at least, my preferred method for entering text into my Freerunner involves using hand writing recognition. i.e. just draw the letters and other symbols on the front of the display and it will Do-The-Right-Thing. In practice, I need an alternate method, at least some of the time.

I do have some code in http://neil.brown.name/git/scribble which does some recognition. However it is only about 80% accurate (if that) and is a little bit slow (possibly because it is written in python). The low accuracy means that I really need to wait for each character to be recognised before starting the next one. And combining this with the slow matching speed means that I actually enter text quite slowly.

I have hopes for making this better and faster. But even then, I suspect that entering punctuation will be a bit of a challenge, and sometimes I just want something more reliable, if a little more awkward in other ways.

So I have a written a little toy that I call 'tapinput'. Its key characteristics are:

For lowercase input, the window looks like this (I've scale down by a factor of two to partially compensate for that fact that the Freerunner has very small pixels).

read more... (No comments)


12 February 2009, 20:54 UTCMoving to Debian on my Neo Freerunner

My Freerunner now runs Debian, which was an educational experience - and education is always a good thing.

But there is a bit of an irony here.

I changed to Debian because of the wealth of packages available. While Open Embedded (which is the base for the FSO distro that I was using) packages lots of stuff, it doesn't package everything. In particular it didn't seem to package python-xlib which I wanted so that I could play with fakekey-like keystroke generation using python code base on pykey by the author of crikey.

So now I have Debian and python-xlib and I am happy. But a problem is that a lot of the toys that people are writing for the Freerunner (like the python EFL Sudoku or the Neon image viewer) are only being packaged as ipk files for the Open Embedded distros. So if I want them I probably have to install by hand or package them myself.

So did I increase the range of available packages by going to Debian, or decrease it?

read more... (No comments)


08 February 2009, 05:22 UTCWhy I wrote my own 'gsmd'

You would think that writing a program to talk dirty to the GSM controller in the Openmoko Freerunner wouldn't be at the top of many peoples TODO lists. After all, it has been done. Multiple times. And having yet another implementation (with doubtlessly a different set of bugs) is just going to hurt interoperability of applications. But I did anyway.

read more... (1 comment)


31 January 2009, 20:51 UTCgsm0710muxd without DBUS or ptys
Since time immemorial, the Hayes "AT" command set has been used for controlling modems and so it is only sensible that controlling modern phones such as GSM devices should also use the AT command set. One problem is that AT is single threaded - while you are on a data call you cannot check signal strength.

So the clever folks who designed the latest version of the "GSM over AT" spec included mutliplexing, so you can have several virtual connections to your phone, one for data, one for control, one for async notifications etc.

This is all implemented quite nicely in gsm0710muxd which is used by Openmoko (and probably elsewhere). But of course it isn't quite as nice as I wanted it. So I've hacked it a bit. My current version can be found at git://neil.brown.name/gsm0710muxd or http://neil.brown.name/git/gsm0710muxd.

There are two things I didn't like. The first is the dependence on DBUS. The second is that insistence on using PTYs to access each channel

read more... (10 comments)


30 January 2009, 21:18 UTCNext Freerunner toys - battery applet and runit

Two more toys that I play with on my Freerunner have just been pushed into git://neil.brown.name/freerunner aka http://neil.brown.name/git/freerunner.

The first is a simply battery monitoring applet. There are plenty of these around and mine adds nothing of real value, except that it is stand-alone and looks the way I want it to look.

It is partly based on the applet in Openmoko Panel Plugin and uses the images for battery status from there. The code (and bugs) are mine though.

The other is a little toy with the creative name of "runit". It will run whatever program it is given and display the output in a window with a button to re-run and another to close the window.

I currently use this for configuring the network (until I get a proper tool for that) and running informational commands like "hcitool scan". It allows me to make important functionality available quickly, while waiting for a more comprehensive tool to be written.

These tools show a significant part of my philosophy which is to create simple stand-alone tools what do what I want, rather the complex frameworks.

One possible problem with this is memory usage. Each python program seems to use up about 5Meg of memory that is resident and not shared. When you only have 128M, that limits you to around 20 such programs at a time. And that doesn't even allow for the kernel.

While I may not want to have 20 running at a time, I am still concerned about the memory wastage. I may end up arranging that python programs are imported rather than executed in a separate process. Python has quite nice namespace control which should make this quite managable, and the gtk.main loop makes is easy to run multiple gtk applications in the one process ... just as long as none of them do any slow processing or call gtk.main_quit.

I'll have to see how that goes....

[permalink (No comments)]


28 January 2009, 02:56 UTCScreen Lock on the Freerunner

(golly, nearly two years since I have posted anything. I probably just think that I have nothing interesting to say...)

About 6 months ago I purchased an Openmoko Neo Freerunner - a nice (if not totally modern) piece of hardware that could make a nice feature-phone if it had working software (which some day it probably will). I've been playing with it on-and-off over the months, though have never used it as a regular phone.

Just lately I have had various bits of leave which have allowed me time to play more intensively with the phone and get it to do some things that it didn't before, or to get it to do them differently. And I thought it might be nice to write about some of that.

Background: I use the "FSO" distribution. However I've discarded illume and e-wm in favour of matchbox-window-manager and matchbox-panel. They might not look as pretty by they seem to be much lighter on resources and they stay quite nicely out of my way. I also don't use "zhone" and have disabled most of "frameworkd"...

All of my programming has been with python using the GTK toolkit for display (via pygtk). I wouldn't say that GTK is a particularly good tool kit (I've used TK before (with TkTcl) and my memory is that it was a much more well rounded kit. But memory could be playing tricks on me). I'm using GTK simply because it was something different but easy to learn.

The first little project I want to talk about and share is my program for locking the screen.

read more... (No comments)





[atom feed]  
[æ]