Archive | Programming RSS feed for this section

Python, Django, & MySQL on Windows 7, Part 3: iPython & Virtual Environments

By on August 28, 2011 in Programming

This is the third post in a dummies guide to getting started with Python, Django, & MySQL on Windows 7.

The last installment covered setting up virtual environments.

Part 3: iPython and Virtual Environments

iPython is not virtual environment-aware

iPython is a valuable tool, but you’ll have to tweak it to work with virtual environments. By default, iPython isn’t aware of a virtual environment’s packages.

For example, if you install a package into a virtualenv and try to use it via Python’s built-in shell, everything works:

import via python

Try the same thing with iPython, however, and you get an import error:

import via ipython

A little intervention is required to make iPython virtual environment-aware. If you’re okay with using Python’s built-in interactive shell instead of iPython, skip ahead to Part 4: Installing Django.

Install iPython to the virtual environment

The easiest and most obvious solution to make iPython work with a virtual environment is to install it to the virtual environment.

However, you will have to do this for each environment you work in.  Furthermore, if you created the virtual environment without the –no-site-packages option (which tells virtualenv not to inherit anything from global site-packages), you may get an “already installed” message.

Modify iPython config file

Another way to ensure iPython behaves with virtual environments is to use its configuration file to check for an active virtual environment and modify the import path accordingly. The directions below assume iPython is installed globally but not in any of your virtual environments.

    1. Open a command window.
    2. Instruct iPython to generate a sample configuration file (called ipython_config.py) by typing ipython profile create
    3. The sample file should now be in your iPython profile folder. On Windows, this is [your user folder]\.ipython\profile_default.
    4. Open ipython_config.py in a text editor and add the following code to the bottom. I relied heavily on code from here and here, making a few tweaks.
      https://gist.github.com/1176035
    5. Activate a virtual environment.
    6. Start up iPython and look for the output confirming the current active virtual environment:ipython & virtualenv
The next installment is Part 4: Installing Django.
Comments { 7 }

Python, Django, & MySQL on Windows 7, Part 2: Virtual Environments

By on August 28, 2011 in Programming

This is the second post in a dummies guide to getting stared with Python, Django, & MySQL on Windows 7.

Part 1 of the Python, Django, and MySQL on Windows 7 tutorial covered the basics:  installing Python and some package management tools.

Set up virtual environments

We saw how easy it is to install Python packages such as iPython.  Just run the install command, and everything you need goes into Python’s site-packages folder.  But what if different projects require different packages or even different versions of the same package?  Of what if you have a project that requires an older version of Python?

If you’re planning to work on multiple applications, you need virtual environments to prevent a big mess of dependencies gone awry. Each virtual environment has its own installation directories, separate from other environments.

Virtual environments are brought to you courtesy of the virtualenv package.  To install the package and create a virtual environment:

  1. Open a command window.
  2. Install virtualenv by typing pip install virtualenv and waiting for the successfully installed virtualenv message.
  3. Create a folder to hold your virtual environments (for example, c:\virtualenvs) and cd it (it’s easier if there are no spaces in the folder name).
  4. Create a new virtual environment by typing virtualenv --no-site-packages [environment name].  The output will look something like this:
    virtualenv output
  5. Look at your virtual environments folder. You should now see a sub-folder with the name of your new environment.
  6. Create another environment, just for fun: virtualenv env2 --no-site-packages
  7. Activate a virtual environment by running its activate script, located in the environment’s Scripts folder. In this example, the command is c:\virtualenvs\env1\scripts\activate.
  8. The command prompt is now preceded by the environment name–that’s how you know the environment is active. The activate script also takes care of adding the virtual environment’s scripts folder to the beginning of your path.
  9. Now that a virtual environment is active, try installing  a package (we’ll use python-twitter as an example): pip install python-twitter
  10. Once the package is installed, you can see that it’s in the site-packages folder of the virtual environment, but the original Python install and the other virtual environments remain untouched.
    module installed into virtual environment
  11. To deactivate the environment, just run the deactivate script in [envrionment name]\scripts\deactivate

Bonus: Virtualenvwrapper is a handy set of extensions for virtual environments. This is a necessary tool if you deal with multiple environments on a daily basis, but it’s not essential right now if you’ll be working in a single environment. The Windows virtualenvwrapper port is here.

Coming next… getting iPython to work with virtual environments.

Comments { 5 }

Python, Django, & MySQL on Windows 7, Part 1: Installing Python, Pip, and iPython

By on August 27, 2011 in Programming

When starting my current job, I was new to Linux, Python, Django, MySQL, and pretty much every other technology the organization uses.  Furthermore, as the office’s only Windows 7 person, I was on my own when setting up a development environment.

I found several helpful tutorials but ended up creating more detailed instructions to reflect my level of experience and our project structure. These notes became my dummies guide to setting up Python, Django, and MySQL on Windows 7.

Install Python

  1. Install Python (I used the Python 2.7.2 Windows Installer).
  2. Modify your path environment variable to include the location of the installed Python executable.  To change environment variables in Windows 7:
    • Click the Windows start button in the lower-left corner of the screen.
    • In the Search programs and files box, type environment variables
    • When the search results appear, click Edit the system environment variablesedit environment variables
    • You should now see the System Properties window.  Click Environment Variables…environment variables
    • When the Environment Variables window opens, choose Path from the System variables list and click Edit…
    • Append the following location of the Python executable and the Python Scripts folder to the variable value, making sure everything is separated by a semicolon. For example, ;C:\Python27;C:\Python27\Scripts
    • Click OK after modifying the variable value, and click OK again to exit the Environment Variables dialog.
  3. You should now be able to bring up an interactive Python shell by opening a command window and typing python.
    welcome to python

Get a package installation and management tool

Many kind people have written functionality-enhancing Python Packages, and sooner rather than later you’ll need to use some of them. Life will be much easier if you have a tool to install and manage these packages.

Two such tools are pip and easy_install (part of setuptools).  Despite some disadvantages on Windows, I chose pip after reading this StackOverflow thread.

Unfortunately, pip doesn’t have a Windows installer. The easiest workaround is to install setuptools (which does have a Windows installer) and then use it to install pip.  Directions below are a modified version of yet another StackOverflow thread.

  1. Download the setuptools MS Windows installer that corresponds to your installed version of Python. If you’re on a 64-bit version of Windows 7, use this instead (thanks to commenter JukkaN for that tip).
  2. Run the setuptools install wizard.  It should automatically find your Python installation directory.
    setuptools install
  3. Make sure you have a way to extract .tar files (e.g., 7-Zip).
  4. Download pip.
  5. Uncompress the pip download
  6. Open a Windows command prompt and cd to the directory that contains the uncompressed pip download (the folder structure looks something like pip-1.1). Make sure it’s the directory that contains a file called setup.py.
  7. From that directory, type python setup.py install
  8. After this script runs, you should see a confirmation message that pip was installed to the site-packages folder of your python install (this folder will house all of the Python packages you install).
  9. If you get an error message, it’s probably because you haven’t added your Python directory to your path (see I above).

Install iPython

You don’t know it yet, but you need iPython.  It’s a richer version Python’s out-of-the-box interactive shell and has some extremely helpful features for people learning the language (object introspection, macros, code completion). iPython is easy to install now that you have a package management tool.

Even though I just had you install pip for this very scenario, it’s not the best choice for installing iPython on Windows. I promise pip will come in handy later, but to install iPython, use setuptools (setuptools will automatically install a module required for iPython’s code completion functionality, but pip will not).

  1. Open a windows command prompt.
  2. Type easy_install ipython
  3. iPython should now download and install.  If you get an error message, make sure your path environment variable contains the Python \Scripts folder.
    ipython install
  4. To use the iPython interactive shell, type ipython from a command window and peruse the built-in documentation.

The next part of Python, Django, & MySQL on Windows 7 will walk you through virtual environments.

Comments { 19 }

Two month milestone

By on May 18, 2011 in Database, Musings, Programming

flowering tree in Childs Park

Monday marked two completed months with the National Priorities Project. Though these weeks haven’t produced much writing, they’ve been a whirlwind of learning:

  • Python
  • Django
  • MySQL
  • The joy of setting up a proper Windows dev environment using the above three items
  • Piston, a tool for powering APIs through Django
  • Linux
  • Git/Github
  • The Federal Budget process
  • The Consolidated Federal Funds Report , a huge annual file of government expenditures.
  • Various other indicators about the state of our union: gas emissions by state, average teacher salaries, people in poverty, insurance enrollments, etc.
  • Finally, I’m NPP’s interim Twitterer, a fascinating distraction.

One day soon I’ll write a Dummies Guide to Setting up Python/Django/MySQL on Windows post. In the meantime, it’s great to be back in the hands-on tech saddle.

Comments { 2 }

Regular Expression for dollar amount validation

By on December 16, 2009 in Programming

I’m finishing up a small project that required validating dollar amounts on web-based form fields, and luckily I was using the wonderful Validation jQuery plugin.

Although this plugin doesn’t come with an out-of-the-box dollar validator method, it’s very easy to add your own. The trick was to find an appropriate regular expression. Although Google turned up several regex examples for validating dollar inputs, none met my exact criteria (the ultimate goal being to provide users with the greatest amount of data-entry leeway):

  • Allow the user to type in the $ (or not)
  • Allow amounts less than one dollar (i.e., 0 – .99)
  • Allow an optional one or two decimal places
  • No negative dollar amounts

So this is what I came up with, and it seems to be working. Two notes: (a) this expression allows inappropriate commas, but I’m stripping those out during the server-side validations, and (b) it is used in conjunction with the max() validator function to ensure the user doesn’t enter anything that could blow up a SQL Server money column:

^\$?[0-9][0-9\,]*(\.\d{1,2})?$|^\$?[\.]([\d][\d]?)$

The final jQUery validator method:

1
2
3
4

//custom validator for money fields
jQuery.validator.addMethod("money", function(value, element) {
	return this.optional(element) || /^\$?[0-9][0-9\,]*(\.\d{1,2})?$|^\$?[\.]([\d][\d]?)$/.test(value);
}, "Please provide a valid dollar amount (up to 2 decimal places)");

Comments { 6 }

Swiz framework for Flex

By on November 21, 2009 in Programming

Swiz sample application

View the sample app code.

Some folks in Wharton Computing, namely Nathan and Adam, have had positive experiences using the Swiz framework for Flex applications.

I haven’t used Flex frameworks in the past because a) I was still learning Flex and wasn’t ready to throw a framework into the mix and b) the only real option was Cairngorm, which seemed like complete overkill.

Now that I’m more comfortable with Flex and understand some of its limitations, however, I can endorse Swiz.

Why do I like Swiz?  Because it doesn’t impose a strict structure and because it solves actual problems.  Even if you don’t care about separating your model from your views or abstracting service calls, Swiz is worth investigating for these reasons:

Event Handling: You can dispatch events at will and add the corresponding event listeners where required.  You don’t have to worry about the fact the Flex events bubble up and not down—Swiz just takes care of it.

Command Chains:  It’s often the case that Flex remote object calls need to occur in a particular sequence or that a series of remote object calls needs to complete before moving on to the next piece of code.  The logic required to ensure everything happens in the right order can get messy and only gets worse over time.  Swiz allows you to define a series of commands (called a CommandChain) and execute them together (sequentially or in parallel).

The Swiz Google code page and the Swiz Framework site have some instructions for getting started with Swiz.  The code samples are piecemeal and outdated, however, so I created a small project that demonstrates basic Swiz functions.  It’s a pretty silly app, but I created it so the Wharton Computing developer community could reference some well-commented code that works in our environment.

Because the back-end services are written in ColdFusion and it’s not especially practical to invest in CF hosting just to demo this, what you see here is just a screenshot* and the code.

Thanks to Nathan, who unknowingly provided some of the code for this sample app.

*Note the Phillies colors on the demo app. The default Flex theme is not meant for public viewing.  Ever.  It’s not hard to spiff up your apps a bit!

Comments are closed

SQL Server operand type clash!

By on February 6, 2009 in Database, Programming

Note:  This article was originally published to the Wharton Computing Developer Center.

Yesterday a fellow developer hit a strange SQL error and determined that the culprit was a T-SQL CASE statement used in his ORDER BY clause.

1
2
3
4
5
6
7
8
9
10

ORDER BY
CASE
WHEN UPPER(@orderBy)='GROUPMESSAGEID' THEN groupMessageID
WHEN UPPER(@orderBy)='GAMEID' THEN gameID
WHEN UPPER(@orderBy)='GROUPID' THEN groupID
WHEN UPPER(@orderBy)='MESSAGEID' THEN messageID
WHEN UPPER(@orderBy)='ROUNDSENT' THEN roundSent
WHEN UPPER(@orderBy)='SENTON' THEN sentOn
ELSE groupMessageID
END

This code results in the following message:
Operand type clash: uniqueidentifier is incompatible with datetime

After some digging around, we found the underlying cause of the error:  the case statement return values have different data types.  In some cases, returning different data types will behave as expected.  However, mixing numeric and character data causes problems.

In the statement above, gameId, groupID, roundSent, and groupMessageID are integers, sentOn is a datetime, and messageID is a uniqueidentifier.  Because the data type precedence pecking order in this case is datetime, int, and then uniqueidentifier, SQL choose datetime as the return type. Uniqueidentifiers cannot be converted to datetimes, hence the error message.

It all became clear after reading this article by George Mastros.  Thank you to George.

Comments are closed

Backtest the musical

By on August 21, 2008 in General Tech, Programming

In September of 2007, the Learning Lab rolled out Backtester,a Flex-based backtesting tool for use in investment classes.  I learned a lot about Flex and ActionScript while working on Backtester and am particularly proud of application’s wizard component, which allows users to design an investment strategy in 3 easy steps and test it against historical data.

We made a short screencast* for last spring’s faculty meeting and have been using it to test video tagging toys such as Viddler and Veotag. This post is just a test of Viddler’s new customizable player.

Comments are closed

ColdFusion: keep it girlie

By on August 8, 2008 in Musings, Programming

Attention, female ColdFusion developers. I found the following information in some recent internet travels:

Silly CF Banner

I, for one, refuse to “upgrade” to a less girlie version of ColdFusion.  In fact, my pores are clogging at the mere thought.  Does anyone have a recipe for scones?

Girlie CF Banner

Comments are closed

HTTPService: passing XML between ColdFusion and Flex

By on June 19, 2008 in Programming

We recently had some trouble with a Flex RemoteObject call. It kept throwing timeout errors on a method that invokes a CFHTTP call, even though we could invoke the same method from a ColdFusion page with no problems.

After increasing the timeout parameters of both the CFHTTP statement and the RemoteObject definition to ridiculous amounts, we decided to use the mx:HTTPService function as an alternative.  Kind of annoying to have to add the extra layer, but it works.

No doubt there is a perfectly logical explanation for the RO timeout, but we needed to get this proof of concept code working as quickly as possible.

The ColdFusion file.
This invokes the method that contains the problematic CFHTTP call and returns the status info in an XML blurb.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

    <cfsetting showdebugoutput=”no” />
 
    <cfset structResult = StructNew() />
    <cfset structResult.backtestId = url.backtestId />
 
    <cffunction name=”RunBacktest” access=”private” output=”false” returntype=”struct” hint=”run a specified backtest”>
 
       <cfinvoke component=”TheProblemComponent” etc />
         …do some stuff….
       <cfreturn structResult />
 
    </cffunction>
 
    <cfset result = RunBacktest() />
    <cfoutput>
    <cfxml variable="resultXML">
       <runResult>
          <backtestResult>
             <backtestId>#result.backtestId#</backtestId>
             <returnCode>#result.returnCode#</returnCode>
             <msg>#result.msg#</msg>
          </backtestResult>
       </runResult>
    </cfxml>
    </cfoutput>
 
    <cfcontent reset =”yes” type=”text/xml; charset=UTF-8″>
 
    <cfoutput>#resultXML#</cfoutput>


The Flex HTTPService definition from Services.mxml:

1
2
3
4
5
6
7
8

<mx:HTTPService id=”runBacktest”
   method="GET"
   resultFormat="e4x"
   result="resultTesty(event);" >
   <mx:request xmlns="">
      <backtestId />
   </mx:request>
</mx:HTTPService>


The result handler
Example of how to reference the XML coming back from CF, even though we ended up not needing to do this.

1
2
3
4
5
6

private function resultTesty (event:ResultEvent):void
{
   var backtestRunResult:XML = new XML(event.currentTarget.lastResult);
   Alert.show(event.currentTarget.request.backtestId);
   do some stuff;
}


Invoking the service call:

1
2
3
4
5
6

private function runBacktest(backtestId:Number):void
{
   Application.application.srv.runBacktest.url = "SuperSecret/RunBacktestFolder/RunBacktest.cfm";
   Application.application.srv.runBacktest.request.backtestId = backtestId;
   Application.application.srv.runBacktest.send();
}

Comments are closed