Saturday, December 11, 2010

Porting DialCentral: Meaningless Results

I'm not sure why I like to wrap up each port with these meaningless results but here we go.

Qt Version
  • Near feature parity with GTK version (no custom callbacks, not re-implemented Alarm UI, no rotation for now)
  • Access to call cancel
  • Divider on various lists (for time ones, tried to balance heavy/light phone users)
  • Condensed history
  • Changed SMS Entry letter count style from GV to Nokia
  • Tabs are cached between launches
  • Display when history/messages tabs where last refreshed
  • 4,866 LOCs
  • From August 18th to December 2nd
GTK Version
  • 7,207 LOCs
  • For forever :)
That is quite an impressive difference in size.   I think a lot of that is owed to a cleaned up design that only needs to worry about Google Voice and not Grand Central.

I've still got some things to clean up, a couple bugs to fix, and improving some UI glitches on Maemo 4.1.  After that my plan is to get all of my applications into the Community OBS so they are available for Meego.

Friday, December 10, 2010

Threading in Dialcentral

So this is a bit more boring of a post.  Pretty much everything went as expected.  The only annoyance  as part of my zeal to avoid concrete inheritance was that Maemo 4.1 seems to have too old of a version of Qt to avoid inheriting from QThread.  That page has some better practice listings for QThread.

DialCentral original used threading to be lazy about startup and to avoid the slowest of operations from blocking the UI thread.  I've now pushed all blocking operations to a thread.  The main question was what approach I would take to  threading in PyQt.  I was tempted to keep using Python threads using GObject callbacks but I keep to one toolkit for simplicity.

In The One Ring I had some code that used generators to trampoline code to the worker thread.  I just had to refactor it so that there were toolkit specific and toolkit agnostic components.

I was glad that python logging works smoothly with QThreads.  The messages get marked with some kind of distinguishing identifier.

See, nice and boring.  No real issue which to me is a plus.

Thursday, December 09, 2010

Latest Lessons on Widgets from Dialcentral

This is a bit of a catch-all post to go back over some previously covered items with some new things learned.

TreeViews


HTML Delegates
I previously couldn't find the way to do custom cell renderers.  In part the oddity is that they provide default ways to render that are meaningless when you provide Delegates.  I did some searching and found an example HTML Delegate.  It had some issues with dark themes so I had to fix several bugs in it.



Simple heirarchy
Next I wanted Dialcentral to have letter separators in the contact list.  I never messed with hierarchical views in GTK before so I can't really compare.  My first time branching out to applying this to Messages and History turned out bad because it seems to have the parent items' columns aligned with the child's columns.  The original History view had 3 columns with the left two scrunched together.  These didn't provide much space for the parent row's text.  When I switched to a condensed history view this became moot.

Condensed history view, allowing the parent text to fit.
When learning about hierarchical views I found out that the root expander decorator is given space by default even if there are no children and it isn't shown.  This saved some space in the UI and will clean up the look of my other applications.

Layouts

The only real comment I have for layouts regards QGridLayout which I've not used before this.  Early on I was looking at using my pie menus for the buttons which had some issues, so I setup the alignment.  This didn't help but when I switched to buttons I eventually found out that it caused the buttons to fit to their minimum size rather than expand.  I probably fixed the pie menu issue without realizing I had thwarted myself.

Combo Box

The only real trouble I ran into with QComboBox was when I wanted it editable.  I was going to use this to allow choosing an existing GV configured callback or a custom one.  On the desktop the only way I've gotten it to insert into the list the item I'm creating is when there is nothing else to capture the enter key press.  Otherwise what gets activated is the "Apply" button (or worst "Clear Account" button when I remove the "Apply" button).  I've not found my way around this issue yet.

Callback Combobox caused problems on desktop Qt

Wednesday, December 08, 2010

Porting Dialcentral to Qt

GrandCentral Dialer
Dialcentral started its life as Grand Central Dialer for the 770.  I got involved by adding support for desktop Linux (which also made possible porting to future versions of Maemo).  Soon after the name change to Dialcentral (out of trademark paranoia) I took over maintaining it.  Google Voice support was added while maintaining Grand Central support as long as possible.  Over time other features were added like texting and group texts.  Eventually I provided an alternate way of interacting with Google Voice through a telepathy connection manager called The One Ring.


DialCentral 1.0.7
This port to Qt has taken longer for many reasons.  The backend has slowly become more compartmentalized over time but the complexity grew faster.  I ended up needing to rewrite everything but the direct wrapper around Google Voice.  Besides some other misc parts of life during this time I was also contacted by a recruiter from a very good company, an opportunity I felt I shouldn't pass up.  My interests have have focused on system programming, programming practices, and software architecture and so I've ended up ignoring algorithms a bit too much.  Their process spread over 2 months which I spent refreshing and expanding my knowledge of algorithms rather than working on Dialcentral.  After a couple trips out seeing the company and the office's city I came to the conclusion that Austin and the company I work for are the place for me to remain for now.  Location was the biggest clincher to me and that was after I researched all of their branch locations and interviewed for one of two I found that I had a chance of liking.

DialCentral 1.2 Beta
I'm still putting some finishing touches on this port.  I'm cautiously testing the waters with releasing this to extras out of concern that Google might break the brittle unofficial API and i have to do an immediate release.  I released a snapshot and have moved on to putting it in extras-devel with dire warnings especially since if an issue arises I might start having the GTK/Qt versions leapfrogging each other (joy of not having PPAs or a stable API).

For me the interesting part is this was my first Qt app that involves threading.  Related to this is that I want to maintain as much common code with The One Ring as possible.  Also with more UI work comes more insight into the better ways of doing things.  I try to keep all of my UIs fairly simple but this is one of the more involved ones.

Sunday, November 21, 2010

Dublin and Meego Conference

Thanks to the generosity of the Linux Foundation I flew out and attended Meego Conference this year as well as payed for a couple extra days to see Dublin.  This is my first time out of the United States (I did pop across the border into western Canada for 10 minutes but I don't think that counts).  I've been a Maemo user since the 770 but I never before attended any of the Maemo Summits.  This was a great experience.

I arrived on Saturday the 13th and left on Friday the 19th.  I ended up site seeing rather than participating in any of the early bird events.

On Day 1 I attended Rob Bradford's location/social aware presentation.  I had not known about libsocialweb before which I thought was interesting.  More interesting is I got some ideas of things to do with geoclue.  I also went to a presentation about the Touch Input Method Framework.  The thing that saddens me about these frameworks being created is that they are in-process and limit you to developing in C++ unless someone goes through the effort for each one to write a proxy plugin.  I try to exclusively deal with Python because it let's me focus more of my time on developing rather than worrying about cross-compiling, debugging, etc.  I can understand that there can be performance issues for some frameworks passing a lot of data over dbus, that was the reason I was given for libfolk.  At the evenings receptions, timeless and I had some interesting conversation.  I just hope I remember all the UI feedback he gave.

On Day 2 I attended the two telepathy related sessions (the project, not the super power though that would have made an interesting and confusing unconference day session).  I am looking forward to telepathy 1.0 despite the extra work it is going to put on me for updating The One Ring (and any future CMs I write).  It will be disappointing if I end up having to drop a couple of platforms (currently I support Maemo 4.1, Maemo 5, and desktop Ubuntu with work being done to prep for Meego).

Some worries of fcrochik and I were eased a bit when Mikhail brought up the QtContact plugin backed by libfolk (a bit before a blogpost hit the various planets).  It still seems odd that QtContacts is built with support for plugins yet you would either need some kind of layer on top to aggregate the plugins (needing retrofits for apps using the old API) or else only have one plugin and it aggregates things that plugin to it (plugin for plugins, joy).

The third day was unconference which is more of a good-chaotic everyone-present-what-they-want kind of day.  Sadly due to a conflict I missed Christian's Wayland presentation.

During one of the slots is when I picked up my Lenovo Ideapad that they were giving away.  A bit weird that it comes with a UK power chord and keyboard (I suspect I'm not the only American that set it to US layout to our confusion).  Sadly there is still some work that needs being done which is not giving people the best perception of the netbook/tablet and of Meego.  Hopefully it'll all be worked out in the thread on f.m.c.

I ended the unconference with leading a session about experiences porting from Maemo to Meego but due to conflicts and being the end of the last day I only had fcrochik in attendance.  We still had the good time.

Being from the US and having seen pretty much no professional football games, I enjoyed the Ireland v Norway game.  I heard from others later that both teams weren't playing all that well.  We had free tickets, the pre-game premium lounge with drinks and pretty good snacks.  We had a good number of rounds of the food because they came in such small bowls.  Thankfully texrat was nearby to take all of our dishes.  I didn't take part much in the half-time or pot-game festivities since the lines were long.

I have to say the venue was great, having the hacker's lounge was great, and it all felt top notch.  Amy, Dawn, Quim, and everyone else involved did a great job and there are many thanks to go around. One lesson learned I think was to have more power (and maybe even power converters at the "Power Plant" for people like me who forgot one).  If my laptop I brought actually had power, it would have been fun to have a day like what I heard happened in Barcelona.

I enjoyed being out in Dublin on the other days.  I managed to do everything by foot besides going from and to the airport (so 10+ km of walking on non-conference days).

On Saturday I spent my time at the city centre.  I saw Marions Square, St Stephen's Green, Dublin Castle, Christ Church Cathedral, and walked Grafton Street.  I enjoyed exploring the city but none of those sites really moved me.  I had some good lamb stew at some place near Dublin Castle.  Coming from Austin and freezing out in Dublin, that stew hit the spot.  That evening I also went and saw Tosca performed at the Gaiety Theatre.  That was a very powerful production.

On the Thursday after the Conference I ended up going to some of the further out spots.  I saw the National Gallery and then walked out to O'Connell Street to see the Spire, the James Joye Statue, the Remembrance Garden (very beautiful both in design and in idea), and the James Joyce Centre.  I then went out west and saw the Leprechaun Museum on my way.  Eventually I was at Kilmainham Gaol and the Ireland Museum of Modern Art.  I then had to rush back real quick to get on the internet to see when I flew back.

Those last three sites ended up being my favorite.  It was fascinating to learn the political history of Ireland from the perspective of a jail, including the execution of the leaders of the Easter Rising.  The Garden of Remembrance was a great place to contemplate.  The Leprechaun Museum was fascinating for learning Irish Mythology.

The things I sadly didn't get to see are the Ireland Botanical Gardens, Mallahide Castle, Knowth/Newgrange, the War Memorial Gardens, and a Famine Memorial I saw on my tour map (if it exists, couldn't find it).  I wasn't even sure how I would get to Knowth/Newgrange but at the Leprechaun Museum they aid that the Dublin Tourism place on O'Connell does a tour up in those parts, you jut need to arrange it a head of time it sounded like.  Something to know for the next trip out to Dublin.

I find it interesting that a friend had recommended I see a lot of the Churches but instead I saw or at least tried to see war memorials, jails, execution sites, ancient burial tombs, etc.  A bit different of priorities I guess.

See Pictures

Wednesday, September 01, 2010

DAQpy

Once upon a time I needed to write an application that interfaced with the real world.  I decided I wanted to use a data acquisition device from NI but I'm picky.  I want easy prototyping with fine control in the finished product.  That kicks out LabVIEW (luckily I'm forgetful or else I could write post after post about the good and bad of LabVIEW).

So I decided to use the C with DAQmx.  For prototyping I decided to use Python with its ctypes library.  I found some example bindings on a scipy page but they were hand generated and didn't seem very pythonic.  So I wrote my own generator and then added some custom helpers on top to make common operations more pythonic.  Unlike pydaqmx (which I just now found out about) I include my code-gen tool and I don't try to create a fake OOP system on top of the DAQmx API.

I even took some of the basic examples included with DAQmx and created the Python equivelants.

You can find a copy of DAQpy on github.  I would caution I re-organized the code on my Linux box which means I don't have access to DAQmx to validate things.  Patches welcome :)

Monday, August 16, 2010

Porting ejpi: Meaningless Results

So let's see how the two versions of ejpi compare:

Qt Version
  • Feature parity with the GTK version
  • Improved pie menu performance but two pie menu display bugs
  • Simplified layout with the potential for rotation
  • 3382 LOCs
  • From June 8th to July 16th
GTK Version
  • 4473 LOCs
  • From January 30th to February 25th
LOCs are measured by "wc -l" rather than getting fancy.  I wrote both apps and they were written in a similar style with similar amounts of whitespace and documentation.


The LOC counts look nice, more drastic than my Gonvert port for what it is worth.  I had a lot more distractions going on during my port of ejpi so a comparison of implementation times between Gtk/Qt or Gonvert/ejpi is as meaningless as the title of this post suggests.

My main Qt disappointment in this port is what seems like high coupling both in Qt itself and in code written by others (the example pie menus).

Now for some of the upsides.  Though I implemented them differently, the similarity in the drawing code between Qt and Cairo helped.  Also finding out about QStandardItemModel made my life easier.

I'm split on which direction I want to go in my learning of Qt.  Originally I had planned on porting Dialcentral for gaining experience with threading.  I have too many ideas though and not enough time.  Possible directions include: porting other apps of mine, a new app using QGraphicsView, or some QtMobility contacts (and eventually calendar) plugins (though then I'd also have to learn the quirks of Qt in C++ and deploying compiled apps to Maemo).

Learning Ruby: Vampire Numbers

Instead of the traditional book group at work I'm in a group where each person chooses to learn a new language.  For me to make it feel worth while it either needs easy C bindings, existing bindings, or be used for scripting at work.  That left me with Go, Lua, Io, and Ruby.  Go was out due to error handling and generics.  I decided to stick with Ruby so it would be work practical (Python is heavily used by the company-wide build tools but my group has a lot of Perl and Ruby scripts).

Our first assignment was simple, identify vampire numbers.  They seem arbitrary and useless but oh well.

One of the first things I did was "gem install rubydoctest".  I like doctests from Python for their providing examples for the code and that they provide very low friction low coverage tests.

My work computer is sadly running Windows.  I did not have tool chains setup to install ruby-prof from source and I couldn't figure out how to install the pre-built version referenced from their site.

A quick summary of my initial thoughts
  • rubydoctest has poor error output but I love that "!!!" drops you into irb (though I didn't find out about it until I was done).
  • I was kind of surprised Ruby doesn't have a factorial function.  I'm not as surprised about a lack of a permutation function, Python's just special like that.
  • Very few examples show how to separate module logic from executable logic (Python's "if __name__ == "__main__" idiom)
  • Optional return's are ugly for most code.  The one place they work well is used inside of a block as a predicate for a function (like the comparator for sorting).
  • Optional parenthesis make it confusing what is a property or method.  Not knowing what is going on feels sloppy to me
  • I got thrown off by changing fairly standard names (exception handling, next/continue) and at first didn't think "next" existed.
  • Find it a strange concept to use next, break, retry, redo inside of blocks.  I'd be curious what the underlying principles are for how the block communicates with its caller to execute those.
  • Retry and redo seem cool though I am unsure how often I would ever use them
  • I like Python's separation of repr from str which makes debug output easy.  I was wanting to figure out what some lists I was using were doing but each element was printed on a separate line, making it take up a lot of space and making it hard to distinguish one array from another.  I didn't notice some of the pretty print functions till later and haven't had a chance to try them yet.
  • My program was slow and according to profiling most of it was in iterating.  (0...5).each was the fastest, next was 0.upto(5), and then last was doing the looping raw (I think).  All of that kind of surprised me, most especially how slow looping was in Ruby.  I was impressed the closures for blocks didn't add much overhead (or at least it isn't noticeable compared to how slow ruby is in general).
  • I kept mixing up whether I was working with an actual iterator and an Enumerable (and it took me a while to notice the difference)
  • I really miss the composable iterators of Python (enumerate plus everything in itertools)
  • Remembering  ".." vs "..." is annoying and I end up having to look it up every time (which is very bad for code readability).
  • A problem that drove me nuts for a while is "x = false or true".  The "=" has higher precedence than "or".  What I needed is "x = false || true".  A blogpost I later came across tries to frame "or" and "and" as flow control operators rather than logical operations. I think there are way too many operators.  Can you name the difference between ".."/"...", "=="/"==="/"eq?"/"equal?", etc?

Thursday, July 29, 2010

Porting ejpi: Tree Models in Qt

For me, one thing that differentiates the quality of a calculator on a mouse or touch driven device is having a calculation history and having it be interactive.  In Gtk I implemented this with a TreeView and a ListStore.  I had never implemented a custom model in Gtk, the included models were simple enough and flexible enough that I didn't have the need.

When I ported Gonvert to Qt, all I could find information for was custom models with QAbstractItemModel and QTreeWidget/QTreeWidgetItem.  QTreeWidgetItem's seem to only be good for Text.  This worked for parts of Gonvert but not for others so I went through the complicated affair of implementing a QAbstractItemModel.  I never felt like I fully understood the relationships of various items which usually worries me.  I also found it slow due to all of the protection I "should" have in my code which causes a lot of calling back and forth between C++ and Python.

I'm unsure how I missed it but when I started on ejpi I came across QStandardItemModel/QStandardItem.  This seems to at least allow a little more flexibility.  I can put icons and checkboxes in as well as text into the model.  I can also attach random python objects to each item.  The downside to a stock model is that you have to externally maintain any invariants but I was already managing those with my Gtk version.

I would be curious about the various performance merits to each approach but I'll have to pass on that for now.  The two main things I found frustrating was finding out about these and how each API was helpful in its own way, sometimes better, sometimes worse than its fellow approaches.

Most of my API issues were due to being unfamiliar with them and having to fight with the different approaches each model system took to presenting itself to the developer.  I still have one unfortunate piece of code in ejpi's calucation history which implements peek through a pop and a push.

I've not messed with setting the font on any of these (I should have for Gonvert and also for ejpi) so I'm unsure how the font experience is in Qt.  Mainly I'm interested in what formatting Pango html-esque tags allowed, changing relative size and basic formatting while keeping everything else to theme defaults.  A quick look through the API seems to suggest this is simple but that is yet to be determined.
I still miss GTK's generic way of handling rendering with its use of CellRenderers.  You don't have to be stuck with the built-in choices for representing items in your model.  It also keeps presentation information away from the model.

I did enjoy how easy QStandardItem's made user edits (sadly only seems to work on the desktop) and activating of individual items.  I didn't get around to exploring drag and drop for history re-arrangement (helpful for RPN calculators) but I don't think I implemented that for the GTK version either.

My GTK version and my Qt version of my ejpi history are available for comparison.

I guess the summary would be that overall I'm a fan of QStandardItemModel and wish I had found it sooner.

Porting ejpi: Icon Theming

ejpi includes some custom icons as well as using theme icons.

The custom icons are used for a couple of buttons.  I know the path to the custom icons and I load them up and everything works as expected.

The stock icons are used for deleting items in the calculator's history as well as a severity indicator and close button for the error info box.

I ported the GTK code from using theme icon names to QIcon.fromTheme.  It was fun for me to find out that fromTheme was added in Qt 4.6 (I still support Maemo 4.1 with my applications and it only has Qt 4.2 I think).  How did applications do this before 4.6?  I don't think I've used KDE since v2 but I think its generally supported icon theming along with everything else which precludes the application building all icons in as resources.

I also had fun with fd.o icon names not being available on all platforms so I had to find alternatives for each and put in fallback code.

I guess chock this up as another thing to abstract away when running on various platforms.

I'm not saying this was hard or anything but just another lessons learned in porting to Qt.

Saturday, July 17, 2010

"Antenna-gate", Fanboys, and Trolls

I just saw a good post on this from the Google Testing Bog.

Despite my dislike of Apple and their treatment of developers (and my disgust that developers are flocking to receive their flogging) I think we should be careful how we react to this.  Problems happen.  There is both good and bad to how Apple handled this. If we feel a bit annoyed in how they handled this we should have that same annoyance no matter the source rather than giving in to fanboy-ism

Do we look at this and think how much better our platform is but get annoyed when others point out the problems with our platform of choice?

This is a general problem whether it be long standing technology flamewars (VI is better btw), to religion, or to politics.  Sometimes the very nature of having two groups forms an automatic antagonism (see also George Washington's Farewell address).

I find Krister Stendahl's rules for religious understanding to be very applicable to all of these areas of our life:

  1. When you are trying to understand another religion, you should ask the adherents of that religion and not its enemies.
  2. Don't compare your best to their worst.
  3. Leave room for "holy envy."

Concurrently Beating a Dead Horse

Sometimes it seems like the horse is already dead but today I came across a fun article describing Erlang's concurrency model.  This is no surprise because Actors (along with STM) seems to be a favorite on the various programming sites.

So why continue the beatings?  I thought I'd solidify my thoughts by writing them out.  I also find exploring these principles beneficial for my projects to improve design even if the high level implementations wouldn't fit within the architecture for some of them.


In any concurrency discussion I think its good to remind ourselves of the pillars of concurrency to help avoid silver-bullet-syndrome.  I highly recommend Herb Sutter's Effective Concurrency series, especially the introductory article where he talks about Callahan's Pillars.
(sorry for the text image, that is how the content was stored in the original article)

STM

First two items of background.  The first is that I'm one of those odd people who enjoys the area where hardware and software meet.  The second is that my current job is in software for test and automation systems.  Having to warn customers about the possibility of Death and Dismemberment is not pleasant.

With those in mind I tend to worry more about how a system, pattern, architecture, whatever talks to the real world than how parallel it can get.  This leads me to dislike STM as my default model of concurrency to use because the world is not transactional.  If you have a program controlling a swinging arm of death, what does a revert mean?  One of Microsoft's researchers on STM.NET admits to this problem and many others (but again it does have its uses).

Dataflow

I enjoyed my experiences with digital design in college but at the moment it is not the field for me.  Also VHDL and Verilog leave much to be desired.

I find the idea of applying the natural concurrency of digital hardware to software fascinating.

In the industry I work in there is a popular data flow programming language.  For high level integration or large datasets it seems great to have the implicit parallelism.  I wonder how much parallelism it really gets with small data sets in simple applications.  The language has a couple draw backs to me for the projects I work on outside of work:
  • Single vendor (never a fan of lock-in)
  • Except for the clunky implementation of Events, it does not handle other concurrency models which a task might be better for.
  • Not designed well for general purpose programming situations
  • Even though some applications seem fast to write the environment feels like it hinders me.
Actors

Now on to actors.  I've always been a fan of the simple concept of handling concurrency through shared-nothing message-passing like Erlang.  I can easily conceptualize how this would work with hardware.  I enjoy the fact that each process is normally short-lived enough that the GC doesn't even need to be called.  These combine to give it soft real-time which is a field I've always found fascinating.

How does it handle various models of concurrency?  I like the idea of having access to an Actor's low level primitives in Erlang to more naturally implement something else like a finite-state-machine.  The low-level message passing is events so you get that.  I don't think I'm doing enough embarrassing parallel tasks to miss having dataflow.  I bet you could model a form of STM for groups of actors with the error system.

Sadly I've never really done anything in Erlang because:
  • At work when I do something in my language of choice I need to optimize for implementation time and not reduced run times which my investigation into Erlang seems to indicate it would be poor at. 
  • With my open source work it usually centers around another component for which I need to have a means of talking to.  Python has a lot of bindings.  I feel like with Erlang I'd have to roll my own.
Actors lose their appeal to me in other languages.  In Python the boilerplate and the overhead to achieve shared-nothing seems too much.  Maybe its just FUD but even in Scala it sounds unappealing.  This leads me to my last item

Events

Events aren't as high-level of a concurrency model as STM and Actors but they can sometimes have a lower barrier to entry, especially when integrating with certain frameworks.  Previously all of my open source applications have been glib based and most were GTK based.  I found glib events a pleasure to work with.

Simple UI callbacks work well.  I could simplify more complex sequences of callbacks like with DBus by abstracting them away but I've tended to abstract the concept so I can do things like map/reduce DBus calls.

Registering idle and timeout events seem to be atomic which makes life great certain applications of parallelism.  What I've tended to do is keep a thread for a group of data (like my connection state for Google Voice).  I then push to a queue a task I want the thread to apply to the data (make a GV call, get contacts, etc) and I include a callback that is registered as an idle callback for when the data is ready.  I have no management of locks to worry about.

I've even been able to abstract away the callbacks to keep good spatial locality on the logic in the code.

An example from The One Ring, my Telepathy Connection Manager for Google Voice:
    @misc_utils.log_exception(_moduleLogger)
    def RequestStreams(self, contactId, streamTypes):
        """
        For org.freedesktop.Telepathy.Channel.Type.StreamedMedia

        @returns [(Stream ID, contact, stream type, stream state, stream direction, pending send flags)]
        """
        contact = self._conn.get_handle_by_id(telepathy.constants.HANDLE_TYPE_CONTACT, contactId)
        assert self.__contactHandle == contact, "%r != %r" % (self.__contactHandle, contact)

        le = gobject_utils.AsyncLinearExecution(self._conn.session.pool, self._call)
        le.start(contact)


        streamId = 0
        streamState = telepathy.constants.MEDIA_STREAM_STATE_CONNECTED
        streamDirection = telepathy.constants.MEDIA_STREAM_DIRECTION_BIDIRECTIONAL
        pendingSendFlags = telepathy.constants.MEDIA_STREAM_PENDING_REMOTE_SEND
        return [(streamId, contact, streamTypes[0], streamState, streamDirection, pendingSendFlags)]

    @misc_utils.log_exception(_moduleLogger)
    def _call(self, contact):
        contactNumber = contact.phoneNumber

        self.__calledNumber = contactNumber
        self.CallStateChanged(self.__contactHandle, telepathy.constants.CHANNEL_CALL_STATE_RINGING)

        try:
            result = yield (
                self._conn.session.backend.call,
                (contactNumber, ),
                {},
            )
        except Exception:
            _moduleLogger.exception("While placing call to %s" % (self.__calledNumber, ))
            return


        self._delayedClose.start(seconds=0)
        self.CallStateChanged(self.__contactHandle, telepathy.constants.CHANNEL_CALL_STATE_FORWARDED)

In the DBus callback (as in, happens in the main-loop) RequestStreams I create an AsyncLinearExecution object (eh, couldn't think of something better for a name) that runs the generator _call as an idle callback (again, executes in the main-loop).  The yield passes a function, args, and kwds to the thread passed into AsyncLinearExecution's __init__ for it to run without blocking the main-loop.  When it finishes it takes the results and passes that out of the yield.  I can even move the exception from the thread to be thrown at the yield.


I maintain AsyncLinearExecution as an object so I can "cancel" it at anytime.  My style of cancellation is for the thread's results to be ignored.   In the try/except this gets represented as a StopIteration exception.  This style of cancellation greatly simplifies my shutdown logic for a Telepathy Connection.

I've not read too much about other event based systems for Python but if they don't have a main-loop I scratch my head as to how they integrate with threading and other features.

In conclusion:
  • STM: Doesn't seem appropriate for any of my applications
  • Dataflow: Digital hardware is fun but not a fan of the main implementation in software
  • Actors: Cool but lack of integration with libraries limits my use of languages designed around it
  • Raw Events: a lot of fun for my open source applications.
I thought I'd also note that we have a fairly young wiki page that touches on threading for Python and one more specifically for PyQt applications.  I've done some work on them on hope to have more information to add to them later as I continue porting my applications to Qt.

Thursday, July 15, 2010

Porting ejpi: Custom Widgets in Qt

Because of my off-repeated distaste for making custom widgets unless needed, the pie menus for ejpi are my first real custom widget.  Learning to write custom widgets is the reason I chose to port ejpi.

I started off with the C++ pie menu code from Nokia's add-ons.  I ported it to Python (I doubt anyone would package the binary and making Python bindings available for Maemo any time soon).  While at it I simplified the code and made it more re-usable.

Qt Pie Menu Design Evolution

QtPieItem is the client-provided object to store everything related to a slice.  The two subclasses are QtPieAction and QtPieMenu.

QtPieMenu works well as a subclass of QtPieItem for when you want submenus but doesn't make sense with the parent QtPieMenu.  What does the weight, icon, and text mean?  Why not use composition instead?  Besides I don't even need sub-menus in my application.  So I dropped this (could easily be added later).

QtPieAction starts to look a lot like QAction.  I figured I could just use QActions for all of my slices with a wrapper to track the weight (slice weight / all slices weight gives the percentage of a circle a slice takes up).

Moving on to QtPieMenu...

The center of the menu and outside the menu cause a cancellation.  I like actions being connected to the center (one of these actions could be a cancel).  I also like to throw the mouse so I wanted an option for infinite outer radius.

Overall I was seeing there were several pieces of code in here that could be reused and built up to make several variants of Pie Menus, depending on what was wanted.  So I broke out pie locations (QPieFiling, basically the Model), rendering (QPieArtist, kind of like a View), and then the actual widget (QPieDisplay, QPieButton, QPieMenu).

This worked really well.  I do not use QPieMenu so oh well.  A QPieButton can have its representation set with a QActionPieItem.


When the QPieButton is clicked it shows a QPieDisplay that is filled up with delicious goodness.


Life Implementing a Custom Widget

This was actually a fairly pleasant experience.

With the original GTK version of ejpi I struggled to grab the current theme's colors but never was able to.  I followed Qt Pie Menu's example in grabbing colors and it actually worked!  There was one bug in the original C++ code, they used a "dark" for unselected text when "mid" is what is called for but luckily Qt actually had great documentation on when to use what color.

The other major challenge on the desktop I had was sizing.  The QtPieMenu only used sizeHint which gives a suggested default size for widgets.  For the buttons I wanted a minimum size of what is needed to draw the item with a suggested size of the client specified size.  I wanted to then listen to resizes and scale my box accordingly so the user would have button separators.

For this I overrode minimumSizeHint and resizedEvent.  I have the suspicion that widgets are supposed to implement sizeHint, minimumSizeHint, and maximumSizeHint rather than setting the sizes directly to leave it in the wisdom of the client to possibly bypass the hints by setting the actual minimum, actual, and maximum sizes.

    def minimumSizeHint(self):
        return self._buttonArtist.centerSize()

    @misc_utils.log_exception(_moduleLogger)
    def resizeEvent(self, resizeEvent):
        self.setButtonRadius(min(resizeEvent.size().width(), resizeEvent.size().height()) / 2 - 1)
        QtGui.QWidget.resizeEvent(self, resizeEvent)

When checking how things ran on Maemo I ran into a couple of strange issues.  So they wouldn't become modal I registered the menus as SplashScreen.  I think the hacking around modality of popups was what killed performance on my GTK version.

Unlike the GTK version I was getting the frost effect to the whole window minus the square around the pie. That provided quite an awkward look.  I didn't find anything for this so I just drop the round pies.

The other problem which is yet to be resolved is that if an edge button is selected the pie appears off-center from the user's click.  The odd part to me is even the middle buttons have the pie go partially off-screen so it must be some kind of how much is it off the screen.

You can find the source code for these pie menus in ejpi's git repo.

Monday, July 12, 2010

Porting ejpi: Dynamic Layouts with (Py)Qt

Previous and Future ejpi UI

The current ejpi layout does not seem very dynamic
In the history column an error info box may appear that you can click away (custom made, not the new one available in the latest versions of GTK).  The "Trigonometry" button brings up a Touch Selector for choosing which secondary keyboard you want visible.

Some problems with this layout:
  • Buttons are oddly shaped
  • The keys would have to be customized for portrait and landscape
  • Showing two keyboards would take up too much space in portrait mode
  • Not enough space left to increase the button text size
Below I created a mockup of the adjusted layout in portrait mode

Showing only one keyboard at a time will allow the buttons to be a bit nicer and gives more space for the history.  Switching the secondary keyboard was two clicks away.  With only one keyboard switches might be more frequent so we'll switch to tabs with icon representations for single click access (despite what I remember of the Maemo guidelines discouraging tabs).  Moving the Push key out makes it available when any keyboard is selected and reduces the need to specialize the keyboard layouts based on orientation.

We've not changed too much in terms of how dynamic the UI is.  The only change is on rotation we will need to switch the keyboards from being to the right of the history to being below the history.

Comparing GTK and Qt Layout systems

GTK's layout system centers around container widgets that provide holes for their children to populate.  Also the HBox are VBox are very static.  Their base class is either abstract or an interface so you have to actually swap out the class in the widget hierarchy to change between a horizontal and vertical layout.

The impression I have of Qt so far is that it separates out the concept of layout from the widget container.  A way to think of it is the widget is the canvas to draw on while the layout constrains the sizes and locations of the things to be drawn.  I actually removed a layout at one point and the widgets stayed visible and in the same location as before.  You can have layout hierarchies but the API  support for layouts is drastically limited as compared to widgets.  Also the separate widget and layout hierarchies can complicate things.

Implementing My Dynamic Layouts

Of course when I got started I didn't understand the difference between the way they did layouts.  I tried to implement everything as if it was GTK.  I had complex layout hierarchies that I was trying to swap out at run time.  It seems simple to change up a flat layout hierarchy but when you are trying to work off of an abstraction the API disparity between widgets and layouts increases the complexity.  I tried multiple approaches to get it all to work to no avail.

I might have come across one possible approach but I ran into the need to do some casting because the API returned QLayoutItem proxies rather than proxies for the derived class QHBoxLayout, etc.  With PyGTK I never had to worry about casting so when hitting it with PyQt I gave up in disgust at how nasty that feels in Python.  Please note that I am a C++ developer by trade and have no problems with it in C++.

It was when I finally connected how layouts work in Qt that I made some progress.  I would create intermediary widgets that I'd assign my layouts to.  I could then add/remove/show/hide them all I wanted without issue.  I think this is the way it was meant to be done.  I think one of the reasons I hit a wall with this as compared to others is my abhorrence for inheriting from QWidget.  In Gonvert I even changed part of my user experience because I could not access certain event information unless I inherited from the Qt widget.

Its kind of hard to do code samples without gutting my entire application and plopping it in here so I'll just provide one class and how it sets up its widget/layout hierarchy:

class QErrorDisplay(object):

    def __init__(self):
        self._messages = []

        icon = QtGui.QIcon.fromTheme("gtk-dialog-error")
        self._severityIcon = icon.pixmap(32, 32)
        self._severityLabel = QtGui.QLabel()
        self._severityLabel.setPixmap(self._severityIcon)

        self._message = QtGui.QLabel()
        self._message.setText("Boo")

        icon = QtGui.QIcon.fromTheme("gtk-close")
        self._closeLabel = QtGui.QPushButton(icon, "")
        self._closeLabel.clicked.connect(self._on_close)

        self._controlLayout = QtGui.QHBoxLayout()
        self._controlLayout.addWidget(self._severityLabel)
        self._controlLayout.addWidget(self._message)
        self._controlLayout.addWidget(self._closeLabel)

        self._topLevelLayout = QtGui.QHBoxLayout()
        self._topLevelLayout.addLayout(self._controlLayout)
        self._widget = QtGui.QWidget()
        self._widget.setLayout(self._topLevelLayout)
        self._hide_message()

    @property
    def toplevel(self):
        return self._widget

    ...

Note the toplevel property.  Kind of nasty but its what I use to insert this into the rest of the class hierarchy

        self._controlLayout = QtGui.QVBoxLayout()
        self._controlLayout.addWidget(self._errorDisplay.toplevel)
        self._controlLayout.addWidget(self._historyView.toplevel)
        self._controlLayout.addLayout(self._userEntryLayout)


Something else neat that I noticed is that in Qt, QHBoxLayout and QVBoxLayout are just used for convenience when working with QBoxLayout which dynamically supports switching from horizontal and vertical layouts.  This made me happy with Qt for once.  I can't imagine there being significance enough of a difference that they should be separate in GTK or if there is that a dynamically switchable layout couldn't be supported.  I can't speak too positively of Qt or people might worry.  I think its a disservice that Qt has QHBoxLayout and QVBoxLayout and recommends their usage.  Do you really need an extra class to not have to type in one extra parameter into the constructor?  Really?

        if maeqt.screen_orientation() == QtCore.Qt.Vertical:
            defaultLayoutOrientation = QtGui.QBoxLayout.TopToBottom
        else:
            defaultLayoutOrientation = QtGui.QBoxLayout.LeftToRight
        self._layout = QtGui.QBoxLayout(defaultLayoutOrientation)
        # Actual handling of rotation with setDirection not implemented yet


Conclusion

QVBoxLayout and QHBoxLayout are your enemy, QBoxLayout is your friend.

Attach layouts to QWidget's to make layout work easy.  It is sad that PyQt requires the use of casting.

I think there is suppose to be some new UI stuff to make dynamic layouts with fancy transitions easy.  As I will talk about when I discuss QTreeView's it is a bit annoying trying to figure out what is the recommended approach for what situation so I just went with (1) what I found documentation for and (2) what seemed familiar.

Porting ejpi from GTK t Qt

"e**(j * pi) + 1 = 0" is the more complete name of this project.  Throwing the "j" in their might be confusing but blame that on my engineering background.  Why yes, I love that fact that Python follows the EE notation over the Mathematical notation.

The idea for ejpi was born from reading a complaint on planet.gnome.org about how computer calculators overly mimic handheld calculators much to their detriment.  On a computer there are a lot more ways to interact with a program than a device with a one-line display and a fixed set of hardware buttons.

So ejpi is designed as an RPN calculator that has an interactive history.  You can delete item and duplicate items.  With a one-line calculator it is easy to forget where you are in a multistep calculation, so ejpi shows you not just the result but the equation that got you there.

A 4" device (n8x0) is limited on what all you can put on the screen and shrinking it down to 3.5" (n900) makes it even harder.  Making matters worse is that even if the device has a qwerty keyboard (n800 doesn't) its not well suited for math.  So I chose to experiment with pie-menus as the keyboard.  Typical pie-menus behave more like a context menu and have the middle of the pie be cancel.  I changed this up a bit to mimic my favorite keyboard for the Palm, MessagEase.  The pies are buttons that if you just click will perform the center's operation.  If you click-drag the pie will popup and then release in the direction of another symbol for it to perform that operation. 

There is a trade-off in how much of the pie's contents the button shows.  If you show too much then it is cluttered.  If you show too little then the user has to explore every button to know what operations are supported and where.

Sadly on Maemo devices, the performance of the pie menus is not what I would like which makes this more of a toy program.  I actually use the HP42 clone more than my own calculator

Unlike when I blogged about Gonvert, I am still working on the ejpi port.  I've got all the basic functionality working except switching keyboards.  I also plan on adding rotation support, directly editing history, some UI tweaks, and maybe one or two other things that crop up.  If the performance of the pies is better I might add an algebraic entry mode to make ejpi more widely appealing.

So far I plan to write about my experience in writing custom widgets for Qt, handling of dynamic layouts, and the various tree models.  We'll see what is left by the time I am finished.

Friday, July 09, 2010

Mordecai

In my previous post I quoted from The Church of Jesus Christ of Latter-day Saints's Articles of Faith.

We believe in being subject to kings, presidents, rulers, and magistrates, in obeying, honoring, and sustaining the law.
I wanted to clarify what this means for when those beliefs are in contradiction to the law that we are meant to sustain.

Some aspects of our belief are encouraged practices and precedence is given to local law, like how we bury the dead.  Others are essential but not contradictory like when laws require a civil marriage to be performed, couples have performed separately their civil marriage and their sealing of their marriage for time and all eternity.  Even though we see the Gospel as being essential to everyone's happiness and eternal salvation, we proselyte only in countries that legally allow it.

Mordecai, Daniel, Shadrach, Meschach, and Abed-nego are great examples for when the Kings and laws contravenes our right to worship.  Rather than rising up in rebellion, Mordecai and the people fasted, prayed, and petitioned the King.  Daniel, Shadrach, Meschach, and Abed-nego all broke the law but honorably faced the consequences of their decision to do that without losing faith.

Please note, these observations are mine alone and are not meant to represent any of the people I am quoting

True Leadership

Around the world last Sunday is marked as July 4th, well, as long as you use the Gregorian calendar.  For the United States it is the celebration of declaring independence from Great Britain.

One of the Articles of Faith that I adhere to is
We believe in being subject to kings, presidents, rulers, and magistrates, in obeying, honoring, and sustaining the law.

This was the focal point of a lesson someone gave recently and we talked a bit about what it means to obey, honor, and sustain the law.

Its amazing what effect the words we chose have on what others say and what we all think.  Someone slipped in a comment about "leaders".  Nothing outright was said but small feelings I got where the conversation led made me worry a bit for two reasons, 1) what are "leaders", and 2) how we view our "leaders".

For (1), as I quoted previously from Hugh Nibley
Leaders are movers and shakers, original, inventive, unpredictable, imaginative, full of surprises that discomfit the enemy in war and the main office in peace. For managers are safe, conservative, predictable, conforming organization men and team players, dedicated to the establishment.
...
For the manager, on the other hand, the idea of equality is repugnant and indeed counterproductive. Where promotion, perks, privilege, and power are the name of the game, awe and reverence for rank is everything, the inspiration and motivation of all good men.
How many of our representatives, executives, and judges are truly leaders?  Personally, it makes me feel dirty calling them "leaders".

Now for (2) which makes me feel even dirtier.  Reading Thomas Paine's comparison of representative governments to monarchical governments in "Rights of Men" has given me cause to reflect.  When reading how people view and treat the monarchical court's I wonder if in part we carried too much of that tradition with us and if we also in part reverted back to previous bad habits?

The elected and nominated officials are our representatives.  We are the United States government, not those people filled in their buildings in Washington.  They are our employees.  Yes, we would love for them to be leaders for that is what is needed to face the challenges they do.  Yes, we need to sustain them, but not as some lord or duke but as underlings that we are to empower and provide needed support and correction to.

Why is it that we use the Peter Principle and Dilbert Principle in offering "promotions"?

Why is it when we have two underlings vying for a promotion that we view it as a competition between the better of two evils? In business when there is no suitable internal candidate for a position, do we still higher one of the unsuitable ones or do we do take the risk of the unknown and perform an external hire to some third party?

Please note, these observations are mine alone and are not meant to represent any of the people I am quoting

Thursday, July 08, 2010

Trust Issues

Where I work we write in C++ but our coding style has been heavily influenced by kernel development and older coding practices.  This means we do not use exceptions, even in userspace.

Rather than using a global variable for errors or peppering our code with return value checks, we pass a reference to an error value into all of our functions.  On an incoming error, the function is suppose to be a no-op.

The group I work in is responsible for shared code by many hardware product teams.  One joys of this is when something a hardware group develops is wanted by others, we become the owners.  Its also fun when they continue to contribute features to it and we remain weak in our knowledge of it.  Oh and we are also clients of some groups that provide shared code for the whole company.

The group underneath us in the stack has had a bug with their tScopedLock code that it can cause a BSOD when an error is being passed in in an ISR.

The code looks something kind of like (changed for simplicity, especially to be free of internal code)

void tClass::operation(void* param, int& errno)
{
  tScopedLock lock(memberLock, errno);

  if(errno != 0)  {
    int tempError = 0;
    cleanup(param, tempError);
    return;
  }
  ...
}

Because we technically own the component they sent the bug report our way with a suggested fix that they tested.

void tClass::operation(void* param, int& errno)
{
  if(errno != 0)
  {

    return;
  }

  tScopedLock lock(memberLock, errno);


  if(errno != 0)  {

    int tempError = 0;
    cleanup(param, tempError);
    return;

  }
  ...

}


This seemed fishy to me.  One issue is the second error check only gets executed if the lock fails to acquire.  Seems like a small scope to worry about cleanup and that should be left for later.  The second is they probably had a reason to do that cleanup and do it under the lock.  I double checked with the implementer and he confirmed it would leak memory.  At least its better than a blue screen.


So my solution is found below.



void tClass::operation(void* param, int& errno)
{
  int tempError = 0;    tScopedLock lock(memberLock, tempError);


  if(errno != 0)   {

    cleanup(param, tempError);     return;

  }
  else
  {
    errno = tempError;
  }
  ...

}


All of the code is irrelevant.  Who introduced the problem and who made the fix isn't relevant.  To me the lesson here is the importance of code ownership (even if you don't know the code you own).  Our clients are under pressure to meet their deadlines and don't feel the same level of responsibility for this code.  It met their immediate needs and they moved on.  We release with them and so have the same deadline but with a smaller bug backlog.  Fair enough we do have a bit less stress.  I still think our ownership of the code helped us to ensure it had the proper quality for the release looming over us.

Another great example is we have code repos for "shared code" that have no owner.  Yes someone originally put it there but the ownership isn't recognized on the business side of things so when you have deadlines coming up it is hard to justify fixes or improvements.  We end up viewing it as the code graveyard.

I'm not saying any of this is novel.  I just think it was a nice clean example, one of those "huh, nice" type of things.

Tuesday, June 29, 2010

Maemo and Android

Lately I've been reflecting on my commitment to Maemo.  Olivier's blog post made some great points in both directions.

Since I like to ramble, how about some background?

I've been a Linux user full-time (at home) for about 8 years.  I've dual-booted for probably another 4 years before that.  I've gone from Slackware to Red Hat (wow, color printing support!) to Debian to Gentoo to Ubuntu.  I remember staying up through the entire night going through Debian's package list one at a time deciding what all amazing pieces of software I wanted installed.

I loved Gentoo's USE flags.  I was importing all of my music collection as Flac.  Debian didn't have Flac enabled for all packages but with USE flags I could enable Flac for everything that could build it in.  I started tiring of Gentoo's multiple day install process.  I would slip up somewhere somehow and wouldn't detect it until the entire system had compiled.  I didn't know what was wrong so I would start over from scratch.  This almost made me abandon Linux.  With Windows everything "just worked".   The "freedom to leave" kept me strong until I decided to "take the wimpy way out" and switch to Ubuntu.  That is the main motivating force behind the software and web apps I use.

For handhelds I originally used palm devices.  More of my data was moving to the web and at college I had wifi everywhere.  I wanted a handheld device that could match my circumstances and wants.  I chose Maemo and have been using it for about 3 years.  It wasn't too long after that that I started contributing to projects.

The lack of polish makes me sometimes wonder about alternatives.  I sadly admit that for a couple months I used an iPod Touch as my 770 was dying and I was hoping the n900 would release soon (it didn't and I got an n810).  That convinced me I couldn't go iOS.  Android is starting to look attractive though.  I've even felt it was the best thing to recommend to some family members who originally followed me onto Maemo.  That was a sad moment.

Some questions I ask myself include:
  • Why should I stay on Maemo when Android is blossoming?
  • Does Maemo have staying power in the market?
  • Would I need to invest as much time in development if I had Android's larger (free) market available?
I then remind myself
  • Working on the projects I do is a great way to develop skills independent of what work projects I do.
  • If I do decide to work on a project, I much prefer VIM + Python to Eclipse + Java. I write my apps on the desktop and validate they transition smoothly to the device.  SSHing into my device is great.  Modifying the source right there and trying things out is great.
  • Maemo has a much better chance as Meego due to becoming decoupled from a single vendor.  I didn't say it would be like Android but it'll be better.
  • The flexibility of the architecture.  DBus is a great system.  I love that Maemo uses telepathy.  After the long process of learning it I was able to integrate Google Voice support both in my phone and my desktop system.  I've started work on a Bluetooth plugin but that has stalled due to other projects.  The types of customizations people develop is great.
So for now I will be sticking to Maemo/Meego.  I hope I keep having reason to stick to the platform.  In the meantime I will keep writing software for it.

Porting Gonvert: Meaningless Results

So let's see how the two versions of Gonvert compare:

Qt Version
  • Feature parity with the GTK version
  • 5 Windows (Two ways of unit conversion, quick jump, and recent)
  • 1985 LOCs
  • From May 26 to June 15th
GTK Version
  • 1 Window
  • 1658 LOCs (766 of that is hildonize.py which Gonvert doesn't use all of it)
  • From October 29th to January 9th
LOCs are measured by "wc -l" rather than getting fancy.  I wrote both apps and they were written in a similar style with similar amounts of whitespace and documentation.

So for all of my criticism of Qt this went fairly quickly without a lot of code. I look forward maintaining this code base going forward.  It is still hard to compare the numbers because of how meaningless they are.  How much should hildonize.py count towards the GTK version?  How much of the GTK development time was me learning the code and providing a smooth refactored path through revision history rather than doing a fresh rewrite?

One nice side benefit of not having to modify the actual unit list or the base conversions is that I'm still shipping the GTK version in case someone really doesn't like the new stuff.  Now that I'm well beyond feature parity I don't think that is as much of a concern anymore.

I've already started on the next phase of my Qt learning process, porting ejpi.  Some outside things have come up which has slowed down my progress.  I'll follow up with posts on that later.

Porting Gonvert: The Actual Port

Starting from scratch made it a bit easier to redo the architecture and pieces of the UI.

I started off with one class named Gonvert that acted as my application class.  It owned all of the global QActions (which I coupled to my settings) and top-level windows.  Ironically enough I did not have it own the QApplication.  I left that to my main function.


I dislike circular references but with Gonvert owning all of the global QActions I passed the instance down into its children so they could add them to their QWidgets and menus.


Gonvert also managed loading and saving of settings.  I normally manage settings manually and then load/save them through Python's ini handling code.  This time I created dictionaries with QAction values being the values.  I then serialized/deserialized using simplejson.  This worked well.


I decided to take better advantage of stacked windows and made Categories a window rather than a dialog.  Like with other GTK applications, I ran into issues with modality and stacked windows when deploying to Maemo so I had to restructure things to allow everything to run on the desktop without being modal.


GTK's way of creating list views and tree views requires a lot of boilerplate but uses minimal inheritance internally and requires minimal inheritance to customize.  I think all you need to inherit for is creating custom cell renderers and creating your on model if you decide to not re-use theirs.  I loved that I could put any data type, including complex python types into the default models.  I never created my own model (though I probably should have in some places).


I must admit, I lied earlier.  I had to use concrete inheritance to do a custom model with Qt.  The models I was seeing were very restrictive and seemed only focused on QStrings.  With Gonvert I need to do math on data in the model and I like to center on the decimal place.  So I had to create a custom model to internally store the raw value and then to have a column for left of the decimal place and a column for the right of the decimal place.  The customizing I did in GTK with CellRenderers seems to be coupled into the Qt model which is really sad.  Why does my model care about presentation?  Why?  Why?  So it looks like for a lot of things I'll have to create custom models.  Sadly creating a model was confusing but since I never did it in GTK I can't say if it is easier or harder.


Gonvert tends to have a lot of units.  For some this is cool in a geeky sort of way.  For others it means they have to scroll through more to find what they want.  I went ahead and implemented favorites.  It worked out pretty easy to write it all.  The favorites editor uses strings only so I took advantage of the built in model.  I originally implemented it by having the user select the row.  This played a tad bit better than GTK with scrolling on Maemo but it was still a bit finicky.  A drag to scroll was sometimes interpreted as a drag to select or unselect.  I eventually switched it to togglable items which got the favorites editor to work smoothly.  I then would have the category and unit selection windows hide the non-favorite rows.  There is a filter to both sort and filter but I had read some bad things about it and just decided to do both manually.


On a related note to scrolling and lists I noticed strange behavior when adding my "Quick Convert" or "Condensed View".  When you drag a list to scroll, the widget fires a selection changed event for the row you started the drag on and then it fires an event to put you back where you were.  Kind of hacky.  I shared code between programmaticly setting a unit type and the user selecting one.  In the programmatic case we want to scroll to the unit.  In the user case this caused their scrolling to jump around because of these selection events firing.


The main view in Gonvert has tended to be a bit slow.  In the Qt version it was even slower especially with Favorites added.  The main view has an edit box where the user types in a value and then below that is a list of all units and what that value gets converted into.  The unit values get updated, and resorted on every keypress.  This with sorting caused odd issues with me setting rows as hidden for favorites so I had to update that every time the user put in a value.  The slow downs:
  • Marked the model as modified more times than needed
  • Modifying setRowHidden more times than needed
  • Doing a lot of calls back into Qt in the innerloop of Qt calling into my code.
When updating all the values I limited re-sorts to only happen when the value column is selected for sort.  I also made it so we would not signal the model as modified first for the modification and then again for the sort.  I also throttled updating of favorites.  When the user edits the value a QTimer gets started if it hasn't already.  At the end of the timer the model gets updated.  This introduces a minor delay in the updating (set to 100ms right now) but reduces the amount of time we block the users editing by good enough amount.  Sadly whenever the timer goes off and the user is still pressing a lot of numbers, there is a slight pause.

Other things
  • I played around with the favorites.
  • I removed a lot of the error checking in my model (like checking to see if the index was valid).  This proved very beneficial (as long as the View is written correctly).
  • I think I also did some other optimizations but those were the main ones.
Yes, I guided myself in this with the use of a profiler.

One nice thing about this port has been that I've not had to touch anything not related to UI like the loading of units and converting a single unit.  Sadly my model before and after has been coupled to the UI.

Monday, June 28, 2010

Porting Gonert: Qt's Design

All of the tutorials probably followed the bad example of Qt for concrete inheritance.  Some features are only available through inheritance.  Others have only recently allowed for composition.

Luckily I was able to get through all of Gonvert without concrete inheritance.  A nice trick to avoid inheriting from QObject to create signals is to hijack QAction.  I actually used QActions for what they were intended in Gonvert but in some future apps I might abuse them.  I find the class hierarchies odd as compared to GTK.  Just one example is GTK encourages inserting widgets into scroll windows but Qt has widgets inherit from them.  I do admit this probably lets them couple the concepts closely enough that it makes it easier to give a native feel on platforms.

The sweet taste of bloat in the morning.  Qt let's you show any QWidget as a window.  This requires less boiletplate for debugging my own custom widgets (which I left for my next porting exercise) but it makes each widget feel bloated that they can completely handle being a window in of themselves.


Qt was designed when C++ had a less extensive set of libraries.  Why does that have to bleed over into the Python bindings though?  When I first dived into GTK I was nervous about my interactions with the underlying types and ownership.  I eventually learned I could just ignore all of it and it felt great.  GTK had a feel that is very native to Python.  Tutorials for PyQt on the other hand have to go into long discussions of the trade-offs between native data types and Qt's data types.  They also had to go over object ownership rules.  This left a bad taste in my mouth.

I also see people recommending Qt equivalents of python objects. I'd rather not couple myself that tightly to the UI framework.  You can keep your database access, your XML, and (I hope) your threading.  That last one I have yet to explore how worth while it is to do.

A minor annoyance, ctrl+c in the shell does not seem to kill Qt.  If things went wrong setting everything up I have to go and kill my application.  That can be annoying for keeping debug cycles short.  I also had it crash on exit a lot.  I have no clue what I changed that removed that.

The old-style signal syntax is nasty in Qt.  The new stuff is pretty good.  I dislike that I don't get a token to use to unregister a slot but I pass in the same function.  This is annoying when your slot is a lambda.  It is also annoying if for some crazy reason you wanted to register a slot multiple times.  Registering multiple times and my use of lambdas brings up another annoyance.  GTK is written in C and so all callbacks had to offer a void* data pointer at the end so anything can be done with them.  In PyGTK this was cool because they exposed that to the user by allowing arbitrary objects to be passed in there (the object was already being passed in when using a bound method).  They even implemented it through *args so you didn't have it bundle up multiple objects yourself, it would do it for you.  In DialCentral I use a single slot for the dialpad and then use the data parameter to differentiate the key.  Qt is designed around slots being on objects I think and is used to member function pointers.  This sadly means they didn't have to have the void* of C days and you have to go through extra steps to pass arbitrary data to a slot.

I do really enjoy QActions.  At least I said something positive, right?  I'll probably have more gripes and more things to praise it for as I continue porting my applications over.

Saturday, June 26, 2010

Porting Gonvert: Qt's Documentation

Why do almost all UI examples rely heavily on concrete inheritance?  As I looked for tutorials, examples, and references, I kept seeing concrete inheritance come up.  It is bad design and it muddles up the examples.  Are they creating a function in the class for their own use or are they overriding a method?  How critical is it they override it to be able to get functionality X?

PyGTK tutorial and reference were great.  Everything is easily found in the reference and almost everything I needed to know was in the tutorial.  Qt's reference is good, PyQt's is a bit more so-so.  As for tutorials I have not found one concrete tutorial that covers 90% of what I need but instead need to jump from tutorial to tutorial depending on the specific thing I am doing.  I've even done searches through source code to try and find how to use somethings but that didn't get me much.

Porting Gonvert from GTK to Qt

On t.m.o I sometimes see requests to port applications from a previous generation of Maemo (like Quicknote and Multilist) or to port a desktop application to Maemo (like Gonvert).

I must be asking for punishment taking on a lot of these.

Gonvert started off as a unit converter for the desktop.  The UI was not well suited for mobile devices.  I had to make the UI more minimal and use my hildonize.py code to make it auto-hildonize.  Underneath the surface I had to do drastic code cleanup to even figure out how all the parts interact (btw if you want to see refactoring in action, check out my early git history with Gonvert).

As part of my plan for learning Qt and preparing for Meego, I decided Gonvert would be my first port.  No custom widgets, no threading, just taking user input and processing it to generate output.


So I'm going to look at Qt's documentation and design, what my experience was like porting Gonvert, and some meaningless statistics on the port.

Sunday, June 20, 2010

On Representation

There is a bastard kind of generosity, which, by being extended to all men, is as fatal to society, on one hand, as the want of true generosity is on the other. A lax manner of administering justice, falsely termed moderation, has a tendency both to dispirit public virtue, and promote the growth of public evils.
Common Sense, Rights of Man, and Other Writings of Thomas Paine
pg 81
Lately (as in over the last year or more) I've been reading various writings of Thomas Paine. In Rights of Man he defends free governments against Edmond Burke's attacks and Paine himself attacks back against monarchy.

How do we compare today with what the propagandist claims us to be on transparency, the role of our "leaders", and taxes:
But the case is, that the representative system diffuses such a body of knowledge throughout a Nation on the subject of Government, as to explode ignorance and preclude imposition. The craft of [royal] courts cannot be acted on that ground. There is no place for mystery, nowhere for it to begin. Those who are not in the representation know as much of the nature of business as those who are. An affection of mysterious importance would there be scouted. Nations can have no secrets; and the secrets of courts, like those of individuals, are always their defects.

In the representative system, the reason for everything must publicly appear. Every man is a proprietor in Government and considers it a necessary part of his business to understand. It concerns his interest, because it affects his property. He examines the cost, and compares it with the advantages, and above all, he does not adopt the slavish custom of following what in other Governments are called LEADERS.

It can only be by blinding the understanding of man, and making him believe that Government is some wonderful mysterious thing, that excessive revenues are obtained. Monarchy is well calculate to ensure this end. It is the popery of Government, a thing kept up to amuse the ignorant and quiet them into taxes.

The Government of a free country, properly speaking, is not in the persons, but in the laws. The enacting of those requires no great expense; and when they are administered the whole of civil Government is performed - the rest is all [royal] court contrivance.
Common Sense, Rights of Man, and Other Writings of Thomas Paine
pg 293
How much mystery do we have in our government? The race to declare negotiations of counterfeiting treaties as state secrets, the convoluting of processes, back room deals, hiding pork in large "public good" bills. How do these things take affect and how do we stop it? I've heard Tocqueville would attribute it to apathy and I believe that would be a good chunk of it. What do we do about it though?

Also look how we inflate the role of public servants from being represenatives, judges, and executors to the great epitaph of Leaders. Most do not deserve that moniker and are at best managers working for their own good. I turn to Hugh Nibley for a good description of the difference between Leaders and Managers
Leaders are movers and shakers, original, inventive, unpredictable, imaginative, full of surprises that discomfit the enemy in war and the main office in peace. For managers are safe, conservative, predictable, conforming organization men and team players, dedicated to the establishment.
...
For the manager, on the other hand, the idea of equality is repugnant and indeed counterproductive. Where promotion, perks, privilege, and power are the name of the game, awe and reverence for rank is everything, the inspiration and motivation of all good men. ...

"If you love me," said the Greatest of all leaders, "you will keep my commandments." "If you know what is good for me," says the manager, "you will keep my commandments, and not make waves." That is why the rise of management always marks the decline of culture.
I highly recommend reading the rest of that commencement address he gave.

Another great quote to contemplate on representative governments accomplishing good:
It is always the interest of a far greater number of people in a Nation to have things right than to let them remain wrong; and when public matters are open to a debate, and the public judgment free it will not decide wrong, unless it decides too hastily.
Common Sense, Rights of Man, and Other Writings of Thomas Paine
pg 301
With managers representing us, we have those who create, pass, and sign laws that contain the worst of both sides to an argument without the benefits of either rather than working from principle and compromising for the greater good. How many politicians have passed laws so hastily that they have not read the very law they are passing? How long will we be hurting from this haste when these laws are such wide sweeping things as covering outrageously sized stimulus plans or national health care?

Thursday, June 17, 2010

From Maemo to Meego

Seek out any project that has one of the following resume-ready words in its description:
  • Multimedia
  • Worldwide
  • Advanced
  • Strategic
  • Revenue
  • Market
  • Technology
  • Rapid
  • Competitive
The Dilbert Principle, pg 81
by Scott Adams
I'm thinking it best to start off with a little background. On talk.maemo.org and throughout the rest of maemo.org I am known as "epage" (I tend not to be too Original). I joined the community when I got myself a Nokia 770 to replace my Palm Pilot I used for keeping myself organized at school. My old college campus has wifi everywhere and I had moved a lot of my data over to online services so I needed a device that could handle my changing needs.

I started off contributing to a project called "GCDialer" a Grand Central client for Maemo. GCDialer would use your browsers cookies file with wget to communicate with Grand Central. This worked great on Maemo but my desktop had already switched to a version of Firefox that changed how cookies were stored. To get this to work on my desktop I took an ActiveState recipe for downloading pages with cookies and integrated it into GCDialer. Since then the other contributors backed off, we switched the name to DialCentral (pre-GV, for trademark concerns), moved over to Maemo 4.1, added a host of features when adding Google Voice support, and added Maemo 5 support.

I'm now the sole maintainer of DialCentral, Quicknote, Multilist, ejpi, Gonvert, and The One Ring. I've also contributed to NQA and flirted with Mer. This list slowly evoled as I've either start new projects (ejpi, The One Ring), I wanted to make improvements to things I use regularly (DialCentral, Quicknote), or people have requested ports (Multilist, Gonvert, NQA).

Recently I've been trying to help out on the wiki with Python Performance and PyQt.

That last one is where I'm going with this. I have a couple projects in the pipeline. One is for a non-profit and I'm waiting on them to give the "Ok" before releasing. Developing an app that was not for my specific needs or the needs of other community members but for a longer term benefit of a non-profit motivated me to finally start preparing for Meego.

With Maemo 6 we were told Qt was the future and GTK was community-support only. I think I saw in the transition from Maemo 6 to Meego that GTK was going to be fully supported. I took that from the Maemo perspective (rather than Moblin) and assumed GTK/Hildon was to be supported. This made me complacent to learning Qt and switching over. It took long enough switching my applications to support Maemo 5 in addition to Maemo 4.1 but to switch the toolkit also?. After some discussion on the mailing list about Hildon on Meego it looks like Stskeeps is going to pull through on this one though I wonder what Hildon is like without the custom GTK underneath.

I still figure I should learn Qt. My plan is the following:
  • Port Gonvert for an intro into the basic Qt UIs
  • Port ejpi to learn about custom widgets
  • Port DialCentral to develop strategies for threading
I've actually finished Gonvert and plan to write up my experience from that. The main reason I finally got around to starting a blog was to provide a source of slightly less outdated information than what I kept running into.