Archive | Programming RSS feed for this section

Python, Django, MySQL & Win 7

By on September 14, 2011 in Musings, Programming

When starting to learn Python and Django, my goal was to set up a robust development environment similar to what we use at National Priorities Project: isolated virtual environments, MySQL, and tools like pip and iPython. Stubbornly, I resolved to make it all work on Windows.

I achieved the goal, but not without a lot pain. If you’re a Windows user getting started with Python/Django, you might have an easier time installing a virtual Linux machine.

Here’s a re-cap of the Windows-specific instructions for installing Python, Django, MySQL, and a few necessary packages and tools.

Parting thoughts:
  • I abandoned the Cygwin approach after running into trouble with Cygwin’s Python install vs the Windows Python install.
  • People have good things to say about ActivePython as a tool to help Python developers to avoid headaches.
Comments { 0 }

Python, Django, & MySQL on Windows 7, Part 5: Installing MySQL

By on September 14, 2011 in Database, Programming

This is the fifth and final post in a  dummies guide to getting stared with Python, Django, & MySQL on Windows 7.

By now, you should have Django installed into a virtual environment.  These tutorials aren’t meant to cover building a django app, just to point out the quirks involved with getting a project up and running on Windows.  These tutorials also assume you want to construct real applications using a real development environment.

To that end, you’ll want a heftier database than sqlite.  We use MySQL at the office, so these instructions cover installing it and using it with Django.

Install MySQL

  1. Download and install MySQL.
  2. Once MySQL is installed, proceed through the configuration wizard. Check Include Bin Directory in Windows PATH box.
  3. When prompted, set a password for the MySQL root account.
  4. Once the installation wizard is done, open a command window and log in to MySQL with the root account: mysql -uroot -p (you’ll be prompted for the password).
  5. After logging in, run the following commands to create a database, create a user for your Django project, and grant the user database access.

Install MySQL-python

You’ll need the MySQL-python package, a Python interface to MySQL.

  1. Download the windows MySQL-python distribution here.  The author has some instructions about the appropriate version; assuming a 32-bit version of Python 2.7, you’d download this package (.exe).
  2. After downloading, do not run the Windows installer. Doing so will install MySQL-python to your root python, which virtual environments created via –no-site-packages won’t be able to see.
  3. Instead, install the downloaded package to your virtual environment by using easy_install, which can install from Windows binary installers:
    easy_install file://c:/users/you/downloads/mysql-python-1.2.3.win32-py2.7.exe (modify to reflect the location of the downloaded installer and its name).installing mysql-python package via easy_install

Configure Django

Next, you’ll need to update the database-related settings of your Django project.

  1. From the directory of your Django project, open settings.py using your favorite editor.
  2. Update the default key in the DATABASES dictionary.  Set ENGINE to django.db.backends.mysql and set NAME, USER, and PASSWORD to the database name, username, and password you chose when installing MySQL.  See Part I of the Django tutorial for more information about database settings.
  3. Open a command window, activate your virtual environment, and change to the directory of your Django project.
  4. Type python manage.py syncdb. This command creates the underlying tables required for your Django project.
    syncdb output
  5. If the syncdb worked, you have Python, Django, and MySQL communicating in harmony.  Congratulations!  You can now proceed through the Django tutorial and create your first application.
Comments { 5 }

Python, Django, & MySQL on Windows 7, Part 4: Installing Django

By on September 14, 2011 in Database, Programming

This is the fourth post in a  dummies guide to getting stared with Python, Django, & MySQL on Windows 7.

We’re finally ready to install Django, a popular Web-development framework. Detailed instructions for building out a Django site are beyond the scope of this humble tutorial; try The Definitive Guide to Django or Django’s online Getting started docs for that.

These directions will simply make sure you can get up and running.

Installing Django

  1. Open a command window.
  2. Go to (or create) the virtual environment you’ll be using for your django project. For this example, I created a virtualenv called django-tutorial: virtualenv django-tutorial --no-site-packages
  3. Install django: pip install django
    install django 
  4. Start an interactive interpreter by typing python (or iPython, if you’ve made it virtual environment-aware).
  5. Test the install by importing the django module and checking its version: https://gist.github.com/1177372
  6. Create a new directory to hold your Django projects and code. Change to it.
  7. Think of a name for your first Django project and create it by running the following command: python -m django-admin startproject [projectname].
    Important: most Django docs show django-admin.py startproject [projectname] to start a new project, which can cause import errors and other trouble for Windows users. See this stackoverflow thread for details.
  8. You should now see the project’s folder in your Django directory:django project folder
  9. Change into the new project folder.
  10. Test the new project by typing python manage.py.  Manage.py is Django’s command line utility; you should see a list of its available subcommands.
  11. A further test is to start up Django’s development server: python manage.py runserver. You should see something like this:
    django runserver

If you’ve made it this far, you’ve successfully installed Django and created your first project.

Next up is Part 5: Installing MySQL.

Comments { 5 }

Python, Django, & MySQL on Windows 7, Part 3: iPython & Virtual Environments

By on August 28, 2011 in Programming

This is the third post in a dummies guide to getting started with Python, Django, & MySQL on Windows 7.

The last installment covered setting up virtual environments.

Part 3: iPython and Virtual Environments

iPython is not virtual environment-aware

iPython is a valuable tool, but you’ll have to tweak it to work with virtual environments. By default, iPython isn’t aware of a virtual environment’s packages.

For example, if you install a package into a virtualenv and try to use it via Python’s built-in shell, everything works:

import via python

Try the same thing with iPython, however, and you get an import error:

import via ipython

A little intervention is required to make iPython virtual environment-aware. If you’re okay with using Python’s built-in interactive shell instead of iPython, skip ahead to Part 4: Installing Django.

Install iPython to the virtual environment

The easiest and most obvious solution to make iPython work with a virtual environment is to install it to the virtual environment.

However, you will have to do this for each environment you work in.  Furthermore, if you created the virtual environment without the –no-site-packages option (which tells virtualenv not to inherit anything from global site-packages), you may get an “already installed” message.

Modify iPython config file

Another way to ensure iPython behaves with virtual environments is to use its configuration file to check for an active virtual environment and modify the import path accordingly. The directions below assume iPython is installed globally but not in any of your virtual environments.

    1. Open a command window.
    2. Instruct iPython to generate a sample configuration file (called ipython_config.py) by typing ipython profile create
    3. The sample file should now be in your iPython profile folder. On Windows, this is [your user folder]\.ipython\profile_default.
    4. Open ipython_config.py in a text editor and add the following code to the bottom. I relied heavily on code from here and here, making a few tweaks.

      https://gist.github.com/1176035

    5. Activate a virtual environment.
    6. Start up iPython and look for the output confirming the current active virtual environment:ipython & virtualenv
The next installment is Part 4: Installing Django.
Comments { 5 }

Python, Django, & MySQL on Windows 7, Part 2: Virtual Environments

By on August 28, 2011 in Programming

This is the second post in a dummies guide to getting stared with Python, Django, & MySQL on Windows 7.

Part 1 of the Python, Django, and MySQL on Windows 7 tutorial covered the basics:  installing Python and some package management tools.

Set up virtual environments

We saw how easy it is to install Python packages such as iPython.  Just run the install command, and everything you need goes into Python’s site-packages folder.  But what if different projects require different packages or even different versions of the same package?  Of what if you have a project that requires an older version of Python?

If you’re planning to work on multiple applications, you need virtual environments to prevent a big mess of dependencies gone awry. Each virtual environment has its own installation directories, separate from other environments.

Virtual environments are brought to you courtesy of the virtualenv package.  To install the package and create a virtual environment:

  1. Open a command window.
  2. Install virtualenv by typing pip install virtualenv and waiting for the successfully installed virtualenv message.
  3. Create a folder to hold your virtual environments (for example, c:\virtualenvs) and cd it (it’s easier if there are no spaces in the folder name).
  4. Create a new virtual environment by typing virtualenv --no-site-packages [environment name].  The output will look something like this:
    virtualenv output
  5. Look at your virtual environments folder. You should now see a sub-folder with the name of your new environment.
  6. Create another environment, just for fun: virtualenv env2 --no-site-packages
  7. Activate a virtual environment by running its activate script, located in the environment’s Scripts folder. In this example, the command is c:\virtualenvs\env1\scripts\activate.
  8. The command prompt is now preceded by the environment name–that’s how you know the environment is active. The activate script also takes care of adding the virtual environment’s scripts folder to the beginning of your path.
  9. Now that a virtual environment is active, try installing  a package (we’ll use python-twitter as an example): pip install python-twitter
  10. Once the package is installed, you can see that it’s in the site-packages folder of the virtual environment, but the original Python install and the other virtual environments remain untouched.
    module installed into virtual environment
  11. To deactivate the environment, just run the deactivate script in [envrionment name]\scripts\deactivate

Bonus: Virtualenvwrapper is a handy set of extensions for virtual environments. This is a necessary tool if you deal with multiple environments on a daily basis, but it’s not essential right now if you’ll be working in a single environment. The Windows virtualenvwrapper port is here.

Coming next… getting iPython to work with virtual environments.

Comments { 5 }

Python, Django, & MySQL on Windows 7, Part 1: Installing Python, Pip, and iPython

By on August 27, 2011 in Programming

When starting my current job, I was new to Linux, Python, Django, MySQL, and pretty much every other technology the organization uses.  Furthermore, as the office’s only Windows 7 person, I was on my own when setting up a development environment.

I found several helpful tutorials but ended up creating more detailed instructions to reflect my level of experience and our project structure. These notes became my dummies guide to setting up Python, Django, and MySQL on Windows 7.

Install Python

  1. Install Python (I used the Python 2.7.2 Windows Installer).
  2. Modify your path environment variable to include the location of the installed Python executable.  To change environment variables in Windows 7:
    • Click the Windows start button in the lower-left corner of the screen.
    • In the Search programs and files box, type environment variables
    • When the search results appear, click Edit the system environment variablesedit environment variables
    • You should now see the System Properties window.  Click Environment Variables…environment variables
    • When the Environment Variables window opens, choose Path from the System variables list and click Edit…
    • Append the following location of the Python executable and the Python Scripts folder to the variable value, making sure everything is separated by a semicolon. For example, ;C:\Python27;C:\Python27\Scripts
    • Click OK after modifying the variable value, and click OK again to exit the Environment Variables dialog.
  3. You should now be able to bring up an interactive Python shell by opening a command window and typing python.
    welcome to python

Get a package installation and management tool

Many kind people have written functionality-enhancing Python Packages, and sooner rather than later you’ll need to use some of them. Life will be much easier if you have a tool to install and manage these packages.

Two such tools are pip and easy_install (part of setuptools).  Despite some disadvantages on Windows, I chose pip after reading this StackOverflow thread.

Unfortunately, pip doesn’t have a Windows installer. The easiest workaround is to install setuptools (which does have a Windows installer) and then use it to install pip.  Directions below are a modified version of yet another StackOverflow thread.

  1. Download the setuptools MS Windows installer that corresponds to your installed version of Python.
  2. Run the setuptools install wizard.  It should automatically find your Python installation directory.setuptools install
  3. Make sure you have a way to extract .tar files (e.g., 7-Zip).
  4. Download pip.
  5. Uncompress the pip download
  6. Open a Windows command prompt and cd to the directory that contains the uncompressed pip download (the folder structure looks something like pip/pip-1.0.x).
  7. From that directory, type python setup.py install
  8. After this script runs, you should see a confirmation message that pip was installed to the site-packages folder of your python install (this folder will house all of the Python packages you install).
  9. If you get an error message, it’s probably because you haven’t added your Python directory to your path (see I above).

Install iPython

You don’t know it yet, but you need iPython.  It’s a richer version Python’s out-of-the-box interactive shell and has some extremely helpful features for people learning the language (object introspection, macros, code completion). iPython is easy to install now that you have a package management tool.

Even though I just had you install pip for this very scenario, it’s not the best choice for installing iPython on Windows. I promise pip will come in handy later, but to install iPython, use setuptools (setuptools will automatically install a module required for iPython’s code completion functionality, but pip will not).

  1. Open a windows command prompt.
  2. Type easy_install ipython
  3. iPython should now download and install.  If you get an error message, make sure your path environment variable contains the Python \Scripts folder.
    ipython install
  4. To use the iPython interactive shell, type ipython from a command window and peruse the built-in documentation.

The next part of Python, Django, & MySQL on Windows 7 will walk you through virtual environments.

Comments { 5 }

Two month milestone

By on May 18, 2011 in Database, Musings, Programming

flowering tree in Childs Park

Monday marked two completed months with the National Priorities Project. Though these weeks haven’t produced much writing, they’ve been a whirlwind of learning:

  • Python
  • Django
  • MySQL
  • The joy of setting up a proper Windows dev environment using the above three items
  • Piston, a tool for powering APIs through Django
  • Linux
  • Git/Github
  • The Federal Budget process
  • The Consolidated Federal Funds Report , a huge annual file of government expenditures.
  • Various other indicators about the state of our union: gas emissions by state, average teacher salaries, people in poverty, insurance enrollments, etc.
  • Finally, I’m NPP’s interim Twitterer, a fascinating distraction.

One day soon I’ll write a Dummies Guide to Setting up Python/Django/MySQL on Windows post. In the meantime, it’s great to be back in the hands-on tech saddle.

Comments { 2 }

Regular Expression for dollar amount validation

By on December 16, 2009 in Programming

I’m finishing up a small project that required validating dollar amounts on web-based form fields, and luckily I was using the wonderful Validation jQuery plugin.

Although this plugin doesn’t come with an out-of-the-box dollar validator method, it’s very easy to add your own. The trick was to find an appropriate regular expression. Although Google turned up several regex examples for validating dollar inputs, none met my exact criteria (the ultimate goal being to provide users with the greatest amount of data-entry leeway):

  • Allow the user to type in the $ (or not)
  • Allow amounts less than one dollar (i.e., 0 – .99)
  • Allow an optional one or two decimal places
  • No negative dollar amounts

So this is what I came up with, and it seems to be working. Two notes: (a) this expression allows inappropriate commas, but I’m stripping those out during the server-side validations, and (b) it is used in conjunction with the max() validator function to ensure the user doesn’t enter anything that could blow up a SQL Server money column:

^\$?[0-9][0-9\,]*(\.\d{1,2})?$|^\$?[\.]([\d][\d]?)$

The final jQUery validator method:

1
2
3
4

//custom validator for money fields
jQuery.validator.addMethod("money", function(value, element) {
	return this.optional(element) || /^\$?[0-9][0-9\,]*(\.\d{1,2})?$|^\$?[\.]([\d][\d]?)$/.test(value);
}, "Please provide a valid dollar amount (up to 2 decimal places)");

Comments { 3 }

Swiz framework for Flex

By on November 21, 2009 in Programming

Swiz sample application

View the sample app code.

Some folks in Wharton Computing, namely Nathan and Adam, have had positive experiences using the Swiz framework for Flex applications.

I haven’t used Flex frameworks in the past because a) I was still learning Flex and wasn’t ready to throw a framework into the mix and b) the only real option was Cairngorm, which seemed like complete overkill.

Now that I’m more comfortable with Flex and understand some of its limitations, however, I can endorse Swiz.

Why do I like Swiz?  Because it doesn’t impose a strict structure and because it solves actual problems.  Even if you don’t care about separating your model from your views or abstracting service calls, Swiz is worth investigating for these reasons:

Event Handling: You can dispatch events at will and add the corresponding event listeners where required.  You don’t have to worry about the fact the Flex events bubble up and not down—Swiz just takes care of it.

Command Chains:  It’s often the case that Flex remote object calls need to occur in a particular sequence or that a series of remote object calls needs to complete before moving on to the next piece of code.  The logic required to ensure everything happens in the right order can get messy and only gets worse over time.  Swiz allows you to define a series of commands (called a CommandChain) and execute them together (sequentially or in parallel).

The Swiz Google code page and the Swiz Framework site have some instructions for getting started with Swiz.  The code samples are piecemeal and outdated, however, so I created a small project that demonstrates basic Swiz functions.  It’s a pretty silly app, but I created it so the Wharton Computing developer community could reference some well-commented code that works in our environment.

Because the back-end services are written in ColdFusion and it’s not especially practical to invest in CF hosting just to demo this, what you see here is just a screenshot* and the code.

Thanks to Nathan, who unknowingly provided some of the code for this sample app.

*Note the Phillies colors on the demo app. The default Flex theme is not meant for public viewing.  Ever.  It’s not hard to spiff up your apps a bit!

Comments are closed

SQL Server operand type clash!

By on February 6, 2009 in Database, Programming

Note:  This article was originally published to the Wharton Computing Developer Center.

Yesterday a fellow developer hit a strange SQL error and determined that the culprit was a T-SQL CASE statement used in his ORDER BY clause.

1
2
3
4
5
6
7
8
9
10

ORDER BY
CASE
WHEN UPPER(@orderBy)='GROUPMESSAGEID' THEN groupMessageID
WHEN UPPER(@orderBy)='GAMEID' THEN gameID
WHEN UPPER(@orderBy)='GROUPID' THEN groupID
WHEN UPPER(@orderBy)='MESSAGEID' THEN messageID
WHEN UPPER(@orderBy)='ROUNDSENT' THEN roundSent
WHEN UPPER(@orderBy)='SENTON' THEN sentOn
ELSE groupMessageID
END

This code results in the following message:
Operand type clash: uniqueidentifier is incompatible with datetime

After some digging around, we found the underlying cause of the error:  the case statement return values have different data types.  In some cases, returning different data types will behave as expected.  However, mixing numeric and character data causes problems.

In the statement above, gameId, groupID, roundSent, and groupMessageID are integers, sentOn is a datetime, and messageID is a uniqueidentifier.  Because the data type precedence pecking order in this case is datetime, int, and then uniqueidentifier, SQL choose datetime as the return type. Uniqueidentifiers cannot be converted to datetimes, hence the error message.

It all became clear after reading this article by George Mastros.  Thank you to George.

Comments are closed