Adding "Remove" function

Not a huge amount of code, but I added an action to delete a run from the tracker.  Basically the run page now has a little trash can next to each run you log.  I found this cool site that has a great page of icons.  I am no design dude, so this was a real life saver.  Selecting the trash can passes the run's BigTable key into the new web action.

class RemoveRunAction(webapp.RequestHandler):
    '''The handler used to remove a Run from the system'''
    def get(self):

        key = self.request.get('key')
        run = Run.get(key)
        run.delete()
        ClearRunCache()
        self.redirect('/')

Like I said, not a lot of code!  Basically get the key from the request, find the run, delete the run, and clear the memcache.  The call I used to clear the cache is memcache.delete which clears the run cache for this one runner only.  It doesn't reset the entire caching system. All the code is updated on github, the links on the left point you to my branch and to the application.  I copy/pasted the code above from github, just noticed it looks a lot better than the copy/paste from eclipse.

Source added to git hub

Motivated by a discussion at work, I added all the source for this app to github.  It is a pretty cool free source code repository.  Free if you want everyone to see your code, pay a few dollars if you want it private.  Very easy setup and configuration.  Feel free to check out my code and offer and comments or suggestions.

http://github.com/jamessummerlin/pyruntracker

You can look at the code right on the site.  They display the code syntax highlighted and with line numbers!

Using Google App Engine memcache

I finally got memcache working correctly with my Google App Engine Model.  I have read where this version of memcache is a bit different than you would expect.  It works just like memcached, but specific to GAE and perhaps a different backend implementation.  Either way, it keeps with memcached APIs and is very easy to use.  I wanted to play with this and see how it works on my test app, the run tracker.  My app's data model is a runner who has entered many runs.  I figured a good use of a cache was to cache the list of runs a runner has entered.

A "Run" is an app engine Model class.  My initial thought was to add that to memcache using this call:

memcache.set(users.get_current_user().email(), runs)

Turns out I made a rookie mistake here.  What I intended to do was add the set of "runs" to the cache with the users email address as the key.  The problem is that it never worked.  I kept getting errors on the template rendering telling me that "'Runs' object is not iterable".  My assumption is that Runs is not a dataset anymore.  Turns out I was correct.  I found this blog post that pointed out my mistake.  I was caching the query and not the data itself.  I had to get the data first

runs = Runs.all().ancestor(runner)

Now this worked, memcache is working on my app!  To get my set of runs from the cache, all I had to do is this:

runs = memcache.get(users.get_current_user().email())

Very easy, no problems after that one speed bump.  I did see something interesting in that blog post I posted above though.  A link at the end of it pointed to this blog.  This is an excellent article on making your memcache more efficient.  I implemented the serialize/deserialize code in my app.  Not sure how much my app will benefit, but it was nice to put in there.

That is it, I need to clean up some code and will post new versions soon.

APUG Presentation

I presented what I have coded so far to the Austin Python Users Group (APUG).  It was a lot of fun.  I was a bit nervous, this was only the second meeting I have attended.  This is a great group of super bright people, so it was nice to bounce this stuff off them and get their input, feedback and brainstorming.  I love that environment.

I have shared by presentation and code through Google Docs.  Feel free to check out what I have done and give me any feedback.

Revision 0.0

Today I sat down to finish up some styling, clean up some code, and publish this test run tracking program to the Google App Engine to see what will happen.  Exciting day!

First I worked on a bit of styling.  If you know me, you know this isn't a strength of mine, but I wanted to at least line things up, give it some color and some font love.  Played CSS for a few hours making the table and forms "presentable".  Still far from a nice looking application, but it will do for now.

Then some code cleanup.  Removed a ton of code that I thought I needed, but turned out I didn't.  It always makes you feel good to remove stuff that was causing bugs anyway.  Then you really feel you have a good clean foundation to start with.  Note a few items.

1. There is no error checking in this version.  The goal was to learn how to go from zero to something that persists data on the Google Application Engine.  Error checking is next :-)
2. No Delete function. That is coming next, before the other next thing.
3. Form fields are just text, no number validation or minutes conversion.
4. Avatars are supported by Gravatar, not the Google profile avatar.  Not sure why Google isn't using their own profile, but I didn't see anyway to get to the Google profile avatar, so I used Gravatar.

Probably a million other items, but those are what is on my list now.  I don't really have a list, this blog is it so I will probably post thoughts on missing pieces as they come up.

So I cleaned up the code, styled it a bit, and it seems ready to go.  Now to get it up on the Google App Engine.  It is a simple procedure really, you register for an app account.  You only get 10 applications, so don't go crazy registering apps.  You give your app a name, and that is it.

Put the application name in your app.yaml file.  appl.yaml needs to live in the root directory of your project, this is what you use to configure you application.  Add your application name here, my app name is "pyruntracker"


application: pyruntracker
version: 1
runtime: python
api_version: 1


Then you use appcfg.py to deploy your project out to Google.  This is the cool part.  Just use the following command line to deploy, appcfg.py is in the app engine SDK.

appcfg.py update directory

Enter your Google email address/password, and off it goes!  It runs a test to see that it is up and running, and that is it.  Super easy, worked perfectly.  Here is the URL to my app

http://pyruntracker.appspot.com/

I am going to post the code too, I just have to figure out a nice way to do that on a blog.  Suggestions are welcome.  Maybe I will create a blog post per file, so I can describe what I did.  There isn't that much code.

Google App Engine Datastore

I spent the better part of the past few days trying to wrap my head around the basics of the app engine datastore and the Python APIs written for it.  It is a bit of a different model than the relational one that I am used to.  More object based, no "table" design means you don't have to translate your objects into tables and then create the relationship tables that connect the various attributes.  This system is driven around your object model and makes a lot of sense.  I just had to drop my long history in relational design and move forward with an open mind.

So here is where you start, the Python Datastore API.  There is a lot of stuff there, I have only scratched the surface.  The GAE model is built on Bigtable, a technology I pointed out a few posts ago.  For a good high level explanation of how the Bigtable model is different from an RDBMS, check out this groovie post.  Like anything, the best way to learn this stuff is just to dive in and do it.  I added persistance using the GAE datastore to my running application.  This is the biggest piece so far.  Here is how I went about it.

Let me review the application I am writing.  A running training log application.  Very simple, login, add your daily run, list runs, logout.  That is it.  So to do this I created these models.

First a runner.  A runner is a subclass of type Model.  It lives in the google.appengine.ext.db package, which means it gets all the app engine persistance goodness included for free.  There are other types of database models to use, but this seemed the easiest for a first try.  My runner includes one property, the user as a UserProperty.  UserProperty is a really cool class that holds a Google user account, built in!


class Runner(db.Model):
    user = db.UserProperty()

I really didn't need a runner model.  You will see when I define a run below, I could have just as well put the runner in that model.  In fact, Bigtable sort of encourages you not to normalize the two classes the way I did.  Still, I kept it this way for a couple of reasons.  One, I wanted to learn how this parent/child grouping works (described below) and Bigtable will do some optimizations on parent/child grouping so this seemed like a good thing to me.  


Here is my run class, again a db Model.  It very simply holds the three properties I am interested in (name of run, distance, duration) and includes a time stamp that is auto created for us.


class Run(db.Model):
    name = db.StringProperty()
    distance = db.StringProperty()
    duration = db.StringProperty()
    date = db.DateTimeProperty(auto_now_add=True)


Simple!  Now the cool part, writing and reading these things.  I wrote a simple web page that provides a form for this data.  After logging in, the user can enter data in the form and submit it.  This is the class that saves the data after submission.


class RunLog(webapp.RequestHandler):
    def post(self):
        
        # Look for the current runner in the datastore first
        runner = Runner.all().filter('user =',users.get_current_user()).get()
        if not runner:
            runner = Runner()
            runner.user = users.get_current_user()
            runner.put()
         
        # Save the run when we have a runner parent
        run = Run(parent=runner)
        run.name = self.request.get('name')
        run.distance = self.request.get('distance')
        run.duration = self.request.get('duration')
        run.put()


        self.redirect('/listRuns')


Note a few things.  I first look to see if this runner has been here and saved runs before.  If so, I use that runner as the "parent" of the run.  This creates an Entity Group and according to what I read is intended to be used with data that is very transactional.  I can use transactions to update and manipulate all the data in this entity group, useful, but not so much for my app.  Again, I wanted to see how this parent/child thing works first, even if it is easier to just include the runner in the run class itself.    If I don't find a current "runner" I will create a new one and save the runs with that runner.


I did read one nice thing about Entity groups.  Apparently Bigtable with provide a bit of an optimization and keep all data related to an Entity group physically close together.  This should mean consistent response times for all data in your Entity group if your users are scattered across the globe.  Seemed cool to me.


Next, reading.  This actually comes first in the web design since the data is presented on the welcome page if you are logged in.  


class MainPage(webapp.RequestHandler):
    def get(self):
        nickname = None
        avatar = None
        run = None
        runner = None
        
        # Check for current user, populate data if found
        if users.get_current_user():
            runner = Runner.all().filter('user =',users.get_current_user()).get()
            if runner:
                run = Run.all().ancestor(runner)
            nickname = users.get_current_user().nickname()
            avatar = gravatar(users.get_current_user().email())
            
        template_values = {
            'runner': nickname,
            'avatar': avatar,
            'runs': run,
            }


        path = os.path.join(os.path.dirname(__file__), 'runs.html')
        self.response.out.write(template.render(path, template_values))


You can see that I am checking for a current user, if so, try to lookup to see if that user is a "runner" in our system.  If this person is a runner, get all of their runs using the "ancestor" call on the Query class that comes back from a call to Run.all().  This will filter on only runs that this runner has entered because we made the parent connection in the RunLog class.  Then I lookup a Gravatar if the user has one, and set the values for the template to render.


It took me a long time to get there, but that seems to me a basic setup.  I feel like I am at about step 1 understanding Bigtable and GAE persistence.  It is a new and different model, so any help is appreciated!

A few tweaks and a gravatar

Two quick problems I solved today that were a bit naging.

When running the app server in Eclipse, it complained I didn't have the Python Image Library (PIL) installed.  Not a huge deal, I'm not going to use PIL now anyway, but I hate runtime errors however innocent they may be.  So I downloaded the PIL source.  Unfortunately I didn't have gcc installed either, so off I go to to find out how to install gcc.  Found an essential developers toolkit for the Mac, Xcode.  This package is great, installs all the dev tools you know and love, and much more.  So with gcc now in place, I built and installed PIL using the instructions in the source.  Went well, and now I don't have an error message when I start the app server.

The second problem was the one I mentioned in the last post.  Everytime I ran the server from within Eclipse, I got this error message.

Variable references empty selection: ${project_loc}

Turns out you need to keep focus on the project as you run it.  So make sure the main project is selected in the package explorer before you execute the Run command.  I suppose this makes sure the ${project_loc} is set.  It sort of makes sense, but is still a little weak.  Thanks to juddsolutions blog post for pointing this out.  It wasn't a Pydev problem after all, we all had to deal with it.


Got a great piece of code from snipt, allowed me to add Gravatars in just about 5 minutes.  It was that easy.


Lastly, if anyone has used django_tables successfully with eclipse and pydev, please let me know.  I installed it correctly, but running the app server from within eclipse throws an error that the library is not found.  It is clearly in the path, both source and runtime (Run) path.  Don't know what is going on.