This site runs best with JavaScript enabled.

Refactoring to Save 4805 Lines of Code


Using Python magic to save a lot of lines of code.

Even after coding in Python for the past five years I've never really considered myself an expert in the language because I find the more I know, the more I know I don't know. I generally keep my code simple on purpose until I have a good reason to be complex - which for most django sites, I haven't had a good reason to be complex.

Today I had good reason. I'm currently building a number of key performance indicator (KPI) stats for Neutron. There are currently 46 different stats that I need to calculate for 5 different time periods.

For each state I need:

  • Stats for start of current day to current time with a comparison to yesterday start of day to the current time.
  • This week compared to last week delta
  • This month compared to last month delta
  • This quarter compare to last quarter delta
  • This year compared to last year delta

I will be building a view for each stat and associated time period to return these values in JSON format. So as it stand there will be 230 views. I needed to come up with something clever to save myself some lines of code. I opted for class based views.

First I built a base class that will return the JSON data in a consistent format:

1class StatWithDelta(BaseDetailView): start = None end = None
2delta_start = None delta_end = None title = None subtitle = None
3
4 def __init__(self):
5 super(StatWithDelta, self).__init__()
6 self.end = djtz.localtime(djtz.now())
7
8 def value(self):
9 raise NotImplementedError
10
11 def delta(self):
12 raise NotImplementedError
13
14 def get(self, request, *args, **kwargs):
15 value = self.value()
16 delta_value = self.delta()
17 try:
18 delta_percent = round((((delta_value - value) / value) * 100), 2)
19 except ZeroDivisionError:
20 delta_percent = 0
21 payload = {
22 'value': value,
23 'delta': delta_percent,
24 'title': self.title,
25 'subtitle': self.subtitle,
26 }
27 return self.render_to_response(payload)
28
29 def render_to_response(self, context):
30 return self.get_json_response(self.convert_context_to_json(context))
31
32 def get_json_response(self, content, **httpresponse_kwargs):
33 return http.HttpResponse(content,
34 content_type='application/json',
35 **httpresponse_kwargs)
36
37 def convert_context_to_json(self, context):
38 return json.dumps(context)

Next I built classes for each required time range. Here is my class for today compared to yesterday:

1class TodayYesterday(StatWithDelta): subtitle = 'Today vs. Yesterday'
2
3 def __init__(self):
4 super(TodayYesterday, self).__init__()
5 self.start = self.end.replace(hour=0, minute=0, second=0, microsecond=0)
6 self.delta_start = self.start - datetime.timedelta(days=1)
7 self.delta_end = self.end - datetime.timedelta(days=1)

Now for each stat I create a class that gets the main value and its delta value. Here is one example:

1class GrossMarginPercent(StatWithDelta): title = 'Gross Margin Percent'
2
3 def value(self):
4 return functions.gross_margin_percent_within(self.start, self.end)
5
6 def delta(self):
7 return functions.gross_margin_percent_within(
8 self.delta_start, self.delta_end)

I thought this was clever, but then I found myself writing a lot of similar code. I would create a class based view for each stat class and time period, then an associated url mapping. So for the stat class above I would have these five classes:

1class GrossMarginPercentDay(GrossMarginPercent, TodayYesterday): pass
2class GrossMarginPercentWeek(GrossMarginPercent, ThisWeekLastWeek): pass
3class GrossMarginPercentMonth(GrossMarginPercent, ThisMonthLastMonth): pass
4class GrossMarginPercentQuarter(GrossMarginPercent, ThisQuarterLastQuarter): pass
5class GrossMarginPercentYear(GrossMarginPercent, ThisYearLastYear): pass

... and these urls:

1url(r'^edu/gmp-dtd/$', GrossMarginPercentDay.as_view()),
2url(r'^edu/gmp-wtd/$', GrossMarginPercentWeek.as_view()),
3url(r'^edu/gmp-mtd/$', GrossMarginPercentMonth.as_view()),
4url(r'^edu/gmp-qtd/$', GrossMarginPercentQuarter.as_view()),
5url(r'^edu/gmp-ytd/\$', GrossMarginPercentYear.as_view()),

You can see the lines of code adding up. I was going to add 230+ lines of code to my urls.py file and 4600 lines of code to my views.py file (20 * 230) following PEP8 guidelines.

So I decided to use one url pattern to send to one view function to dynamically create each of the stat-period classes. Here is my new url pattern:

1url(r'^(?P<category>[\w\-]+)/(?P<period>day|week|month|quarter|year)/'
2r'(?P<base_class_name>\w+)/\$', 'magic_view'),

And here is my "magicview" function that where the _magic happens:

1def magic_view(request, category, period, base_class_name):
2 """
3 Builds a dynamic class subclassing the base class name passed in and a time period class. It will return its as_view() method.
4
5 URL structure: /category/period/KPI_Class/
6
7 category: KPI category (edu, conversion, etc.) not really used at this point
8 period: day, week, month, quarter, year
9 KPI Class: One of the class names in this file
10 """
11 class_name = '{}{}'.format(base_class_name, period.capitalize())
12 _module = sys.modules[__name__]
13 base_cls = getattr(_module, base_class_name)
14 if period == 'day':
15 period_name = 'TodayYesterday'
16 else:
17 period_name = 'This{0}Last{0}'.format(period.capitalize())
18 period_cls = getattr(_module, period_name)
19
20 # Create a dynamic class based on the base class and time period class
21 cls = type(class_name, (base_cls, period_cls), dict())
22 return cls.as_view()(request)

So if you include all the comments lines to explain why I did, I'm only using 25 lines of code to save 4830 lines. That's a lot of typing. Python, my fingers thank you!

Discuss on TwitterEdit post on GitHub

Share article
Dustin Davis

Dustin Davis is a software engineer, people manager, hacker, and entreprenuer. He loves to develop systems and automation. He lives with his wife and five kids in Utah.

Join the Newsletter



Dustin Davis