Strategies for Getting the Right Feedback at the Right Time

Our Starting Point: The Overburdened Loan Model

Imagine the Loan model as a bustling city center: it's crowded, chaotic, and handling way more than it was designed for. This model is no different. It acts like a monolithic structure, struggling under the weight of its multiple responsibilities. Take a look:

class Loan(models.Model):
    borrower = ForeignKey(Borrower)
    beneficiary = ForeignKey(Student)
    guarantor = ForeignKey(Guarantor)
    principal = MoneyField()
    loan_product = ForeignKey(LoanProuduct)
    
    # Underwriting functions
    def underwrite(self): ...

    # Documents to be generated
    def generate_contract(self): ...
    def generate_disclosure_statement(self): ...

    # Monitoring functions
    def get_dpd_status(self): ...
    def get_behavioral_score(self): ...
    
    # Email notification functions
    def send_repayment_reminder(self): ...
    def send_payment_acknowledgement(self): ...

The code above is merely an outline, a folded version of what is a behemoth in reality—imagine that each method expands to dozens or even hundreds of lines of code. Collectively, we're talking about a file that can easily exceed 2,000 lines.

The model is responsible for underwriting loans, generating various types of documents, monitoring loan performance, and even managing email notifications. This amalgamation of tasks makes it difficult to manage, and its extensive list of dependencies can be paralyzing.

In essence, our Loan model is like a jack-of-all-trades but a master of none. It's cumbersome to navigate, prone to errors, and a real challenge for any developer to work with. We need a strategy to disentangle this Gordian knot of code.

Step-by-Step: Extracting the Document Generation Concerns

We can see a trend in our model. It is concerned with three categories of behaviors: underwriting, document generation, and monitoring. Let's extract the concern of document generation:

Step 1: Identify Methods for Extraction

First, pinpoint the methods within your model related to document generation. In the case of the Loan model, these methods could be generate_contract and generate_disclosure_statement.

Step 2: Create the Docgen Base Class

Create a new file, app/docgens.py, and within it, define the Docgen class. This will act as the base class for all your document generators.

# app/docgens.py
class Docgen(object):
    template_name = None

    def __init__(self, subject):
        self.subject = subject

    def get_template_name(self):
        return self.template_name

    def get_context(self):
        return {}

    def generate_file(self):
        # File generation logic
        pass

Step 3: Subclass for Specific Documents

Create specific subclasses for each document you wish to generate. In your case, you'll want a LoanContractDocgen and a LoanDisclosureDocgen.

# loans/docgens.py
from app.docgens import Docgen

class LoanContractDocgen(Docgen):
    template_name = "contracts/loan_contract.html"

    def get_context(self):
        # Build context specific to Loan Contract
        pass

class LoanDisclosureDocgen(Docgen):
    template_name = "contracts/loan_disclosure.html"

Step 4: Update Views

Update your views to utilize the new Docgen concern. Remove the old methods from the Loan model, and update your view functions.

# loans/views.py
from .docgens import LoanContractDocgen

def generate_contract(request, loan_id):
    loan = get_object_or_404(Loan, id=loan_id)
    loan_contract = LoanContractDocgen(loan)
    loan.contract_file.set_file(loan_contract.generate_file())
    loan.save()

Step 5: Move Dependencies

Move any dependencies that are only required for document generation into app/docgens.py. This helps isolate dependencies related to each concern.

We still have a few concerns left inside our Loan model, and it's now your turn to extract them. Let me give you some guiding principles to help you proceed.

Extracting Model Concerns: Guiding Principles

Guiding Principle #1: Models are Stable Building Blocks

Rather than feeding individual attributes to the concern, just pass the entire model instance. Attributes may change over time, but the model itself is a stable interface.

# Instead of this
docgen = LoanContractDocgen(loan.principal, loan.borrower, loan.guarantor)

# Do this
docgen = LoanContractDocgen(loan)

Guiding Principle #2: Models are the Core Subject

Concerns operate on data, and the most coherent place to pull this data from is the model itself. Think of concerns as machines that use the model as an input.

# Monitoring concern that uses Loan model as its subject
monitor = LoanMonitor(loan)
dpd_status = monitor.get_dpd_status()

Guiding Principle #3: Keep Concerns Out of Models

Once you've removed a concern from a model, don't put it back. Use the concerns where they are needed.

# Don't do this
class Loan(models.Model):
    def underwrite(self):
        underwriter = Underwriter(self)
        underwriter.underwrite()

# Do this
def process_underwriting(request, loan_id):
    loan = get_object_or_404(Loan, id=loan_id)
    underwriter = Underwriter(loan)
    underwriter.underwrite()

Guiding Principle #4: Concerns are Not a Novelty

The concept of extracting concerns is nothing new. It aligns with established patterns like forms, serializers, and querysets.

# serializers.py - Concern for serialization
class LoanSerializer(serializers.ModelSerializer):
    class Meta:
        model = Loan
        fields = '__all__'

# forms.py - Concern for form handling
class LoanForm(forms.ModelForm):
    class Meta:
        model = Loan
        fields = '__all__'

Guiding Principle #5: Encapsulation Over Exposure

Prefer to encapsulate logic within concerns, rather than exposing model attributes and methods. This way, you don't accidentally manipulate model data in ways that are inconsistent with the business logic.

# Instead of directly modifying model attributes
loan.status = "Approved"

# Use a concern to encapsulate this logic
underwriter = Underwriter(loan)
underwriter.approve()

Guiding Principle #6: Make Concerns Stateless

Concerns should be stateless, meaning they should not maintain state between function calls. This makes the code easier to reason about and test.

# Don't store state in the concern
class BadUnderwriter(object):
    def __init__(self, loan):
        self.loan = loan
        self.is_approved = False  # Stateful

# Do make it stateless
class GoodUnderwriter(object):
    def __init__(self, loan):
        self.loan = loan
    
    def approve(self):
        self.loan.status = "Approved"

Guiding Principle #7: Use Composition Over Inheritance

When you need to share behavior across different concerns, use composition rather than inheritance. This will allow you to mix and match functionalities more freely.

class DisclosureWithSignature(LoanDisclosureDocgen):
    def get_context(self):
        context = super().get_context()
        context['signature'] = SignatureComponent().get_signature(self.loan)
        return context

By incorporating these guiding principles into your development practices, not only do you enhance the maintainability and reliability of your codebase, but you also pave the way for a more sustainable and scalable software architecture.

Django's framework is already well-architected and highly extensible, but when you integrate the concept of concerns, you add another layer of scalability and extensibility. You'll find that navigating and extending your code becomes a more manageable endeavor, and new team members can more easily understand the architecture

I haven't tended for a bonsai tree—but I can imagine writing abstractions to be like it. Trim until they're pleasing to the eyes. Remove the unwanted parts. Grow it to elegance.

In this post, I'll share some properties of code abstractions I found to be pleasing to the eyes.

Reflect the abstract into concrete

class LendingContract(models.Model)
class LendingContractDocGen(PDFDocGen)

class Investor(models.Model)
class InvestorContractDocGen(PDFDocGen)

class InvestorDeposit(models.Model)
class InvestorWithdraw(models.Model)

Without seeing the implementation details, we can see that we're working with "entities". These entities, while each have their own responsibilities, all work together. Sub-systems that create a large system.

A LendingContract stored in the database abstracts an actual legal agreement in the real world. An Investor stored in the database holds the record of an individual involved in the business.

Entities are swappable. Good structures allow that.

class LendingContractDocGenV2(PDFDocGen):
	template = "contracts/lending_contract_v2.html"
    
# lenders/views.py
def generate_contract_file(request, id):
    ...
    # Replaced the contract used in the controller
    contract.file = LendingContractDocGenV2(contract).\
    get_encrypted_file(contract.passphrase)

In the example above, I declared LendingContractDocGenV2 then swapped out the previously set LendingContractDocGen with it.

You could even take it up a notch by making the generate_contract_file() class-based and setting the DocGen as a class-variable:

class GenerateContractView(View):
    # swap LendingContractDocGenV2 here
    contract_docgen = LendingContractDocGen
 
    def post(self, request):
        contract = LendingContract.objects.get(id=id)
        contract.file = self.contract_docgen(contract)\
            .get_encrypted_file(contract.passphrase)
        contract.file.save()
        contract.save()

Hide away sub-system implementation details

# app/docgen.py

import pdfkit
from django.template.loader import render_to_string

class PDFDocGen(object):
	template = "docgens/default_template.html"
    context = {}
    
    def __init__(self, subject):
    	self.subject = subject
        
    def get_template(self):
    	return self.template
            
    def get_context_data(self):
    	return context

    # XXX: Complex bits start here
    def get_file(self):
        context = self.get_context()
        template = self.get_template()

        html_contents = render_to_string(template, context)
        pdf_contents = pdfkit.from_string(
            input=html_contents,
            output_path=None,
            options=self.get_layout_settings()
        )
        return io.BytesIO(pdf_contents)

    def get_encrypted_file(self, passphrase):
        from PyPDF2 import PdfFileReader, PdfFileWriter

        file = self.get_file()
        input_pdf = PdfFileReader(file)
        output_pdf = PdfFileWriter()
        output_pdf.append_pages_from_reader(input_pdf)
        output_pdf.encrypt(passphrase)
        with io.BytesIO() as output_stream:
            output_pdf.write(output_stream)
            output_stream.seek(0)
            return output_stream.read()

Sub-system complexity should be hidden away from the system implementation.

In the code example above, document generation PDFDocGen is a sub-system that has a complex function that produces a pdf file from a template PDFDocGen.get_file() and another that produces an encrypted one PDFDocGen.get_encrypted_file(passphrase). This is irrelevant to the lending system we're building. We know we want to create encrypted PDF files, but the algorithm is irrelevant. We might need to reuse the sub-system in other areas of the system also (lender contracts, bond contracts, non-disclosure agreements, etc) As such, this detail should be abstracted.

One would write a utility function, then create partial functions for reusability. It would look like this:

# apps/utils.py
def generate_pdf_file(template, filename, context):
    # Do complex bits here...
    
# lending/utils.py
lending_contract_pdf_file = partial(
    generate_pdf_file,
    "lending_contract.pdf",
    "contracts/lending_contract.html"
)

While it works—reusable and extensible, I found it less beautiful to this alternative:

# lending/docgen.py
class LendingContractDocGen(PDFDocGen):
    template = "contracts/lending_contract.html"
    filename = "lending_contract.pdf"
    
    def get_context_data(self):
        return {
            'signer_name': self.subject.signer_name,
        }
        
# investors/docgen.py
class InvestorContractDocGen(PDFDocGen):
    template = "contracts/investor_contract.html"
    filename = "investor_contract.pdf"
    
    def get_context_data(self):
        return {
            'signer_name': self.subject.signer_name,
        }

Make system implementation behavior declarative

# lending/models.py
class LendingContract(models.Model):
    signer_name = models.CharField(max_length=255)
    passphrase = models.CharField(max_length=255)
    file = models.FileField()
    
# lending/docgen.py
class LendingContractDocGen(PDFDocGen):
    template = "contracts/lending_contract.html"
    filename = "lending_contract.pdf"
    
    def get_context_data(self):
        return {
            'signer_name': self.subject.signer_name,
        }

# lending/views.py
def generate_contract_file(request, id):
    contract = LendingContract.objects.get(id=id)
    contract.file = LendingContractDocGen(contract)\
        .get_encrypted_file(contract.passphrase)
    contract.file.save()
    contract.save()

Good abstractions encourage you to write system behavior more declaratively than imperatively. While there would still be instances of imperative code, it would be comprehensible that it explains how "entities" interact with each other.

The example above ties how the abstraction sits in between the Model and the Controller (or called "views.py" in Django-land).

The model LendingContract declares the fields it has and how it maps to the database table.

The docgen LendingContractDocGen declares with the template details and how overrides how it generates the context loaded into the template.

Finally, the generate_contract_file() controller  "controls the flow" of the interaction between these two objects. When it is called, it does exactly what it's supposed to do—it generates the lending contract and saves it into the LendingContract table.

System control flow should always be in controllers.

I hope you found value in this post! Post in the comments section below how you write trim your code.

While I was only able to cover one pattern (Prototype pattern) in the examples above, there's a lot more you could apply in your practice—you can find them here: Refactoring Guru. It distills all the patterns described in the book written by the Gang of Four (look it up).

Web on expansion

Web is both public and private: Some content is indexable by crawlers while some are behind auth walls. Everyone who has access to the internet has access to the web (accessible). While it has a lower hurdle for acquisition, it does not provide the same level of capture as a mobile app would have.

"Remember icanhascheeseburger? What was the URL again?" -- It's easy to forget URLs of websites you don't visit often. Hopefully you can Google it again (importance of SEO).

Building on the web is great for growing your top of the funnel, from Awareness to Conversion. By making your content public, you reach a wider audience -- even if it is not something that they don't need now, it might be something that they would need in the future. You are building Mindshare.

Mobile on retention

It provides you a deeper capture on the users for as long as they have the app. You get to nudge (in-app notifications) them at almost no cost. However, it has a taller hurdle for acquisition, whatever is in the app must be valuable enough for the user to install the app (commitment); and valuable enough for the user to keep it (personal data).

There's a stigma on social apps where they access your data and sell it to others. It's at the back of all of our heads. But we install it anyway, because we want/need that app.

Getting the user to install the app is the first step (warm lead). It signals intent that they want to use/transact-with your app. The bottom portion of the marketing funnel, Transaction to Loyalty, takes play.

Both distribution channels can benefit from content play

For mobile/private-web, we can draw inspiration from Patreon (user-generated/various), Masterclass (published/educational), Netflix (published), Spotify (music/vetted), Headspace (in-house/utility).

For public/semi-public web, we can draw inspiration from Medium (user-generated/various), NYTimes (published/news), Youtube (video/user-generated), Head-Fi (user-generated/niche forum).

Great if you can build both at once, but...

Sometimes, resources are constrained and you can only build one first. Pick one goal (expansion or retention) then run with it.

I'll be writing life, software engineering, and other things of interest. Stay tuned!