Top 5 Untrends According To Me
My dear friend Ochronus posted an article titled Top 5 trends and technologies in software development that got me thinking. My thoughts below. Go check Ochronus’s blog if you haven’t, he is the lead developer at Arukereso.hu.
I agree with the suggestions from the original article. Yet, I would like to change the order a little bit; DVCS and then agile (with lowercase a) and then the rest. None of my points below are cool trends, in fact I can guarantee most of you will find them boring. But I think they are all important. OK, I hope you are all psyched now. Here we go:
1. Be Careful With The Buzz
Trends are cool. What could be wrong about following cutting edge stuff? We all want to be up to date, no? I think it’s good to follow the trends if you have the experience and the ability to filter the BS. I know a young developer who was constantly going back and forth between Rails/Ruby and Django/Python. I haven’t heard from him for a while, but he is probably still doing that same dance. Why? Because his considerations were solely based on buzz, not on simple requirements analysis or technical comparisons or personal experience.
2. Learn And Use An Old-Fashioned Low-Level Language
To all the scripting people, like me, out there: you need to have an understanding of what’s happening under the hood. At the least to appreciate our high-level environments, at the most to become genuinely good programmers. Being a Python person myself, I think the best low-level language to be proficient for me is C. Many other high-level languages have C interfaces. So investing the time to learn C should pay off one way or the other.
3. Do Less Web Programming
Aren’t we doing a lot of web programming these days? Actually I think doing X development exclusively is bad for your programming muscles. Web programming, enterprise work or system scripting, it doesn’t matter. But web programming happens more than anything else. Maybe some of you have only been playing with it, but there are a huge number of us doing nothing but web programming. This is so sad; both in an individual level and for the community at large.
4. Learn How To Educate Yourself
What is a noob? Here is a definition and disambiguation (from newbie):
Newbs are those who are new to some task and are very beginner at it, possibly a little overconfident about it, but they are willing to learn and fix their errors to move out of that stage. n00bs, on the other hand, know little and have no will to learn any more. They expect people to do the work for them and then expect to get praised about it, and make up a unique species of their own.
Make an active effort not to be a noob. Learn how to ask smart questions, how to communicate others and seek help. Being polite is good but actually improving and being a valuable member of the community is much, much better.
5. Open Source Properly
It’s great to open source your project. But please do it properly. There are already too many unmaintained, undocumented projects out there that noone seem to care. Do you really have to add to that? As is argument doesn’t make much sense today. But if you really have to make an open source dead drop, please at least document the status of your project and your intentions clearly.
I wouldn’t be surprised if some you think they all are obvious. But if they are so obvious then why are they widely being ignored? Is it because they are under-retweeted, under-reddited and therefore not trendy.
Discrimination
I have wealthy friends, I have friends who are doing worse than me. Most of my friends are university graduates, some of them barely finished primary school. I don’t pay too much attention of their attractiveness but some of them ought to be more beautiful than others. Some of my friends speak with a clear, good accent while others have strong eastern accents. You might find the way they speak funny. But I don’t. I make friends with people who are streetwise and resourceful. You might call some other friends of mine “mother’s darling”. Of all these things I have mentioned above; none of them factor in how close a friend is to me. I wouldn’t go so far as to say I don’t care. But they are definitely not determining factors for my friendships. I don’t discriminate people based on their social status, education, appearance, etc.
My relationship doesn’t go very far with somebody who is not a man of his word. My close friends are punctual, reliable and they keep their promises. I don’t hang out with snobs, envious, scornful people. But the most important qualities I look for in a friend is being cultivated and refined and open minded. I am attracted to people who has a spark in their eyes. You know, the opposite of being a zombie. I make very strong distinctions on these matters. Everybody wants to have nice friends. I’m a nazi about it. I’d rather not waste any time on shallow people and their silly affairs.
We live in a society where any and all kinds of discrimination is frowned upon. I call this “but he is a good guy” disorder. That but clearly indicates there is something not good about that guy, he is not all-good. Scrape any negative attribute off of people. Are we all equally honest? Are we all self-sacrificing and understanding? And then that guy is actually a very nice person. Very nice my ass.
Discrimination based on gender or color or whatever superficial feature is bad. Even kindergarden children know that. Not discriminating at all is idiotic at best and bad for the society at worst. You have to make a choice between people, you can’t be intimate with everybody. You better not base your choice on stupid criteria like coolness or popularity or, god forbid, assertiveness. This is the exact same mistake we make choosing our rulers… ehm, I mean political representatives.
This is not just about human beings. We need to pick out the good from bad when we are making spending decisions. I have seen Food, Inc. recently and it talks about this (near the end). Corporations hide behind many layers of obscurity and we don’t easily see how they are doing their business. To compound that we, as individuals, think ourselves as too small to matter against big companies. But the truth is the only thing we can do happens to be a very powerful way to send them a message. If a company is evil, don’t buy its product. You can’t do much else anyway, but this one act should be enough.
So, make a choice. Or not. Choice is yours.
Developing Reusable Django Apps: App Settings
First part is here: Developing Reusable Django Apps.
Eventually your app will need some sort of configuration. Supplying many parameters to customise your views, template tags and filters to allow template authors easily harness the power of your app and implementing registration pattern are all sensible things to do. But at some point you will need configuration for your app at project level. Because app level configuration is not reusable. Consumers of your app should never have to change its source code. And what better place to put our configuration statements than settings.py? Remember; we want to make things easier for our app’s consumers, not harder. There is no need to add a new file to the project for a few lines of settings[1].
We already know we shouldn’t import settings.py directly. Instead we import the settings object (much like a singleton) of django.conf module:
from django.conf import settings
Now you can access different configuration options as attributes on this object. But I suggest you to use getattr() in order to avoid getting AttributeErrors. Also, notice how we didn’t hardcode the name of the attribute in the second method below:
# This is too verbose
try:
some_setting = settings.SOME_SETTING
except AttributeError:
some_setting = DEFAULT_VALUE
# Plain and simple
some_setting = getattr(settings, 'SOME_SETTING', DEFAULT_VALUE)
Instead of requiring consumers to define all your app settings it is better to supply sensible defaults. Also I find it useful to prefix names of app settings within settings.py
# in settings.py
MYAPP_FOO_CHOICES = [('bar', u'Bar'), ('baz', u'Baz')]
# in myapp/models.py
from django.db import models
from django.conf import settings
FOO_CHOICES = getattr(settings, 'MYAPP_FOO_CHOICES', [('quux', u'Quux')])
class FooRecord(models.Model):
foo = models.CharField(max_length=10, choices=FOO_CHOICES)
This works fine for simple apps with fewer settings. But it can easily get out of hand when your app grows. An app_settings.py module would help keeping track of configuration by keeping all configuration options in one place:
# in myapp/app_settings.py
from django.conf import settings
FOO_CHOICES = getattr(settings, 'MYAPP_FOO_CHOICES', [('quux', u'Quux')])
# in myapp/models.py
from django.db import models
from app_settings import FOO_CHOICES
class FooRecord(models.Model):
foo = models.CharField(max_length=10, choices=FOO_CHOICES)
To summarize the points above:
- Import settings from django.conf
- Use getattr()
- Always supply a default value.
- Prefix settings you made up in settings.py
- Use app_settings.py if you have many
Next part is Developing Reusable Django Apps: Signals.
| [1] | If your configuration is long, say more than 100 lines, you should step back and reconsider. Perhaps you should prefer a strategy similar to django.contrib.sitemaps or django.contrib.syndication. |
Quest For Ultimate Development/Deployment Toolset: Fabric, Pip & Virtualenv
I stumbled upon this post following my frustration with setuptools after virtualenvising telvee repository. I have played with buildout to create repeatable deployments before. Don’t get me wrong, buildout is superb. But I have a few minor issues with it:
- Buildout can create an interpreter but the isolation from the host system is not a good as virtualenv. (Please correct me if I’m wrong)
- Buildout is a dependency whereas virtualenv is a system-wide tool.
- It is, naturally, more work to set it up than a couple of ad-hoc bash scripts.
None of these issues make buildout less awesome, I am just not comfortable enough with it. I am considering other alternatives and their strengths and weaknesses.
Virtualenv
I am quite comfortable with virtualenv. It creates an isolated Python environment for you. For instance the following command will create an environment inside test directory that doesn’t have access to packages installed system-wide:
~/$ virtualenv --no-site-packages test
That is of course when you activate the virtual environment with:
~/$ cd test
~/test/$ source bin/activate
(test)~/test/$
Note the (test) prefix to your prompt. Any packages you install within this environment will be installed only for itself and won’t be available system-wide. If you have omitted --no-site-packages argument you could have access to globally installed packages too. When you are done with this virtual environment you can issue deactivate command to return to your normal shell. That’s basically what virtualenv does.
You can script virtualenv to a certain extend:
#!/usr/bin/env bash
virtualenv $1
cd $1
source bin/activate
echo $PATH
echo $PYTHONPATH
Or you can create bootstrap scripts with virtualenv. Virtualenv doesn’t provide a clean and powerful enough API here, just a callback and two methods to modify commandline arguments.
Virtualenv is great at what it does. But I think setuptools as a package installer (and it’s just an installer, not a manager) cripples virtualenv. Good news is newer versions have a commandline argument --distribute that appereantly substitutes distribute for setuptools.
Pip
My experiments with pip went just fine. Except I bumped into this problem. Globally installed pip was seeing system-wide packages when used on a virtual environment created with --no-site-packages. Again, good news is it is fixed in the trunk.
I will play with pip more once figure out how to best integrate the these applications.
Fabric
I see fabric as the glue that binds everything together. I have known about it long before, but I never had the chance to experiment. I have read most of the documentation2 and played with it a little. Basically the following script is an attempt to create a virtual environment and install pip and django on it, much like the shell script above:
from __future__ import with_statement
import os
from fabric.api import *
from fabric import context_managers
def _get_virtualenv_location():
location = prompt('New location: ', default='../test')
env.envdir, env.envname = os.path.split(os.path.abspath(location))
env.envpath = os.path.join(env.envdir, env.envname)
print('using "%(envname)s" at "%(envdir)s" as virtualenv' % env)
def _virtualenv(command):
with context_managers.cd(env.envpath):
result = local('. bin/activate && ' + command)
return result
def clone():
_get_virtualenv_location()
with context_managers.cd(env.envdir):
local('virtualenv --no-site-packages --clear %(envname)s' % env)
print _virtualenv('echo $PATH')
_virtualenv('easy_install pip')
print _virtualenv('pip install django==1.1.1')
It took me a while to figure out how I can issue commands within a virtual environment. Since fabric commands don’t share state sourcing activate has no effect on subsequent commands. This SO entry helped me to write _virtualenv() function. It is kind of ugly making all functions but fab commands private. I think if fab used __all__ or something similar it would be more explicit. Also a contrib module for virtualenv would be nice3.
Fabric has cool features such as failure handling and code/config editing. It is a great tool to create repeatable deployments. It feels great to be coding in python (well, to some extend). Perhaps the resulting code is a little too complex for local operations. I wish I could write a fab command that handles both local and remote deployments. I think it is not unusual to deploy on the same machine in another location. But having to SSH into localhost is weird, don’t you think?
I am sure there are better ways to accomplish the goal of the script above. Maybe there is an entirely different way to integrage fabric, virtualenv and pip. So, comments and suggestions are welcome as usual.
Maybe I’ll revisit buildout again as well.
1: You need to bootstrap (install) buildout with each development/deployment site.
2: About one third of it I think.
3: I would happily attempt one, once I learn fabric a little better.
Developing Reusable Django Apps
Django app structure is an implementation of seperation of concerns. It is slightly different than what you can find in other MVC frameworks. The stack is split vertically, not horizontally. And then the app is split horizontally within, i.e. models, views, templates etc are in their seperate modules/packages/directories. This vertical splitting allows you to collect all ingredients of one functionality in your project in one place.
Framework Structure
I think apps are one of the strong points of Django. A selling point if you like. There is a great ecosystem of apps, you can find an app for almost anything posssible with Python. And Python is kick-ass when it comes to library wealth. But there is another major advantage of apps when they’re done right; a sane code base. Here is a slide from the Django in the Real World presentation by Jacob Kaplan-Moss:
The fivefold path
- Do one thing, and do it well.
- Don’t be afraid of multiple apps.
- Write for flexibility.
- Build to distribute.
- Extend carefully.
I will focus on flexibility and interoperability of apps in this post. But before we proceed I would like to emphasize the first bullet point in the slide. Because the scope of your app plays a big role in its flexibility and interoperability. Apps should be small enough to easily understand and integrate (into a project). Many times I have moved away from an otherwise good app because of its many dependencies and/or excessive features. On the other hand apps should be big enough to allow for different configurations and allow extension without modifying their code. Do one thing, and do it well.
Scope of an app
Take django-tagging for example; it’s 1.3 KLOC but it does tagging and nothing else. There are no dependencies other than Django, you can add tags to any model without modifying the model source, a tag can be associated with any type of model and tagging hides the gory details from you… In short; finding the right size is important. This is why tagging is the tagging app for Django.
Building For Reuse
General advice is “even project specific apps should be reusable”. Slapping the same app onto another project is not the only advantage. In fact it may not be possible if you are not in the habit of upgrading your whole project to recent versions of Django. The main advantage as I have said before is sanity. I prefer Django to other web frameworks/environments because it provides a civilized way of development. Let’s accept it; web programming is not a particularly interesting, exciting or intellectually rewarding field. You write the same piece of code over and over. And worst of all the challanges you face are actually a result of either the underlying system was designed by morons or you are trying to use it for something it’s not intended to be used. So it is only natural that web programmers feel they’re rusting. Django eases the pain. If you stick to certain conventions serenity will follow as well.
Naturally the framework does most of the work regarding app flexibility and interoperability. Take URLs for instance include('myapp.urls') and you are good to go. You don’t have to bind views one by one. Is it inflexible? Who said urls.py can only contain a hardcoded list of URLs. You can do anything that is possible with Python. You can generate different urlpatterns based on a setting for instance.
It is relatively easy and straightforward to reuse and extend forms and views (both function based and class based). Models are a little harder to get right though. You should always think of the most difficult situation which is you can’t touch either app’s code. Registration pattern of admin app provides a good solution here. You can register a third party model to another third party app in just a few lines.
You don’t need to write lots of code to get the flexibility and interoperability. Well designed apps make good use of settings.py for example. Why should the project developer wrap a view when a single line assignment would do the job? Supplying good templatetags and template snipplets (includes) is another way to make things easy for app consumers.
Signals provide a great way to propagate the events generated from your app. Even though they are one way2, signals are extremely powerful. Any number of observers can connect to a signal and you can send a signal anywhere in your code. Literally. It is even possible your app suppying a signal and then another app sending it3.
There are many more ways to tame your app to be reusable. It all starts with your determination and discipline. Just like documentation, testing and maintaining a software generally. I will write more about reusable apps.
Next part is Developing Reusable Django Apps: App Settings.
1: The word application is used both for a web application and a Django application. To avoid confusion I always use app to indicate the latter.
2: Signals don’t have return values. But you can use a callback AFAIK.
3: I can’t think of an example this would be useful, but still…