Refactoring to Save 4805 Lines of Code
Photo by Stefan Steinbauer on Unsplash
Using Python magic to save a lot of lines of code.
Even after coding in Python for the past five years I've never really considered myself an expert in the language because I find the more I know, the more I know I don't know. I generally keep my code simple on purpose until I have a good reason to be complex - which for most django sites, I haven't had a good reason to be complex.
Today I had good reason. I'm currently building a number of key performance indicator (KPI) stats for Neutron. There are currently 46 different stats that I need to calculate for 5 different time periods.
For each state I need:
- Stats for start of current day to current time with a comparison to yesterday start of day to the current time.
- This week compared to last week delta
- This month compared to last month delta
- This quarter compare to last quarter delta
- This year compared to last year delta
I will be building a view for each stat and associated time period to return these values in JSON format. So as it stand there will be 230 views. I needed to come up with something clever to save myself some lines of code. I opted for class based views.
First I built a base class that will return the JSON data in a consistent format:
1class StatWithDelta(BaseDetailView): start = None end = None2delta_start = None delta_end = None title = None subtitle = None34 def __init__(self):5 super(StatWithDelta, self).__init__()6 self.end = djtz.localtime(djtz.now())78 def value(self):9 raise NotImplementedError1011 def delta(self):12 raise NotImplementedError1314 def get(self, request, *args, **kwargs):15 value = self.value()16 delta_value = self.delta()17 try:18 delta_percent = round((((delta_value - value) / value) * 100), 2)19 except ZeroDivisionError:20 delta_percent = 021 payload = {22 'value': value,23 'delta': delta_percent,24 'title': self.title,25 'subtitle': self.subtitle,26 }27 return self.render_to_response(payload)2829 def render_to_response(self, context):30 return self.get_json_response(self.convert_context_to_json(context))3132 def get_json_response(self, content, **httpresponse_kwargs):33 return http.HttpResponse(content,34 content_type='application/json',35 **httpresponse_kwargs)3637 def convert_context_to_json(self, context):38 return json.dumps(context)
Next I built classes for each required time range. Here is my class for today compared to yesterday:
1class TodayYesterday(StatWithDelta): subtitle = 'Today vs. Yesterday'23 def __init__(self):4 super(TodayYesterday, self).__init__()5 self.start = self.end.replace(hour=0, minute=0, second=0, microsecond=0)6 self.delta_start = self.start - datetime.timedelta(days=1)7 self.delta_end = self.end - datetime.timedelta(days=1)
Now for each stat I create a class that gets the main value and its delta value. Here is one example:
1class GrossMarginPercent(StatWithDelta): title = 'Gross Margin Percent'23 def value(self):4 return functions.gross_margin_percent_within(self.start, self.end)56 def delta(self):7 return functions.gross_margin_percent_within(8 self.delta_start, self.delta_end)
I thought this was clever, but then I found myself writing a lot of similar code. I would create a class based view for each stat class and time period, then an associated url mapping. So for the stat class above I would have these five classes:
1class GrossMarginPercentDay(GrossMarginPercent, TodayYesterday): pass2class GrossMarginPercentWeek(GrossMarginPercent, ThisWeekLastWeek): pass3class GrossMarginPercentMonth(GrossMarginPercent, ThisMonthLastMonth): pass4class GrossMarginPercentQuarter(GrossMarginPercent, ThisQuarterLastQuarter): pass5class GrossMarginPercentYear(GrossMarginPercent, ThisYearLastYear): pass
... and these urls:
1url(r'^edu/gmp-dtd/$', GrossMarginPercentDay.as_view()),2url(r'^edu/gmp-wtd/$', GrossMarginPercentWeek.as_view()),3url(r'^edu/gmp-mtd/$', GrossMarginPercentMonth.as_view()),4url(r'^edu/gmp-qtd/$', GrossMarginPercentQuarter.as_view()),5url(r'^edu/gmp-ytd/\$', GrossMarginPercentYear.as_view()),
You can see the lines of code adding up. I was going to add 230+ lines of code to my urls.py file and 4600 lines of code to my views.py file (20 * 230) following PEP8 guidelines.
So I decided to use one url pattern to send to one view function to dynamically create each of the stat-period classes. Here is my new url pattern:
1url(r'^(?P<category>[\w\-]+)/(?P<period>day|week|month|quarter|year)/'2r'(?P<base_class_name>\w+)/\$', 'magic_view'),
And here is my "magicview" function that where the _magic happens:
1def magic_view(request, category, period, base_class_name):2 """3 Builds a dynamic class subclassing the base class name passed in and a time period class. It will return its as_view() method.45 URL structure: /category/period/KPI_Class/67 category: KPI category (edu, conversion, etc.) not really used at this point8 period: day, week, month, quarter, year9 KPI Class: One of the class names in this file10 """11 class_name = '{}{}'.format(base_class_name, period.capitalize())12 _module = sys.modules[__name__]13 base_cls = getattr(_module, base_class_name)14 if period == 'day':15 period_name = 'TodayYesterday'16 else:17 period_name = 'This{0}Last{0}'.format(period.capitalize())18 period_cls = getattr(_module, period_name)1920 # Create a dynamic class based on the base class and time period class21 cls = type(class_name, (base_cls, period_cls), dict())22 return cls.as_view()(request)
So if you include all the comments lines to explain why I did, I'm only using 25 lines of code to save 4830 lines. That's a lot of typing. Python, my fingers thank you!