Scaling Conversational AI: Designing Chat Interfaces for Financial Services

The AI/ML Challenge

Deep dive into conversational AI financial services chatbot — lessons from building BizStreet at TechSaaS.

<div style="margin:2.5rem auto;max-width:600px;width:100%;text-align:center;"><svg viewBox="0 0 600 200" xmlns="http://www.w3.org/2000/svg" style="width:100%;height:auto;"><rect width="600" height="200" rx="12" fill="#1a1a2e"/><text x="80" y="25" text-anchor="middle" fill="#94a3b8" font-size="10" font-family="system-ui">Input</text><circle cx="80" cy="50" r="14" fill="none" stroke="#3b82f6" stroke-width="2"/><circle cx="80" cy="100" r="14" fill="none" stroke="#3b82f6" stroke-width="2"/><circle cx="80" cy="150" r="14" fill="none" stroke="#3b82f6" stroke-width="2"/><text x="230" y="25" text-anchor="middle" fill="#94a3b8" font-size="10" font-family="system-ui">Hidden</text><circle cx="230" cy="45" r="14" fill="#6366f1" opacity="0.8"/><circle cx="230" cy="85" r="14" fill="#6366f1" opacity="0.8"/><circle cx="230" cy="125" r="14" fill="#6366f1" opacity="0.8"/><circle cx="230" cy="165" r="14" fill="#6366f1" opacity="0.8"/><text x="380" y="25" text-anchor="middle" fill="#94a3b8" font-size="10" font-family="system-ui">Hidden</text><circle cx="380" cy="55" r="14" fill="#a855f7" opacity="0.8"/><circle cx="380" cy="100" r="14" fill="#a855f7" opacity="0.8"/><circle cx="380" cy="145" r="14" fill="#a855f7" opacity="0.8"/><text x="520" y="25" text-anchor="middle" fill="#94a3b8" font-size="10" font-family="system-ui">Output</text><circle cx="520" cy="80" r="14" fill="none" stroke="#2dd4bf" stroke-width="2"/><circle cx="520" cy="130" r="14" fill="none" stroke="#2dd4bf" stroke-width="2"/><line x1="94" y1="50" x2="216" y2="45" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="94" y1="50" x2="216" y2="85" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="94" y1="50" x2="216" y2="125" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="94" y1="50" x2="216" y2="165" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="94" y1="100" x2="216" y2="45" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="94" y1="100" x2="216" y2="85" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="94" y1="100" x2="216" y2="125" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="94" y1="100" x2="216" y2="165" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="94" y1="150" x2="216" y2="45" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="94" y1="150" x2="216" y2="85" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="94" y1="150" x2="216" y2="125" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="94" y1="150" x2="216" y2="165" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="244" y1="45" x2="366" y2="55" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="244" y1="45" x2="366" y2="100" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="244" y1="45" x2="366" y2="145" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="244" y1="85" x2="366" y2="55" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="244" y1="85" x2="366" y2="100" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="244" y1="85" x2="366" y2="145" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="244" y1="125" x2="366" y2="55" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="244" y1="125" x2="366" y2="100" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="244" y1="125" x2="366" y2="145" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="244" y1="165" x2="366" y2="55" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="244" y1="165" x2="366" y2="100" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="244" y1="165" x2="366" y2="145" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="394" y1="55" x2="506" y2="80" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="394" y1="55" x2="506" y2="130" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="394" y1="100" x2="506" y2="80" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="394" y1="100" x2="506" y2="130" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="394" y1="145" x2="506" y2="80" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/><line x1="394" y1="145" x2="506" y2="130" stroke="#e2e8f0" stroke-width="0.5" opacity="0.3"/></svg><p style="margin-top:0.75rem;font-size:0.85rem;color:#94a3b8;font-style:italic;line-height:1.4;">Neural network architecture: data flows through input, hidden, and output layers.</p></div>

At TechSaaS, we deploy AI models that serve real users — from Skillety's recruitment matching to our PADC memory system with hybrid BM25+vector retrieval.

In this article, we'll dive deep into the practical aspects of scaling conversational ai: designing chat interfaces for financial services, sharing real code, real numbers, and real lessons from production.

Model Architecture & Selection

When we first tackled this challenge, we evaluated several approaches. The key factors were:

We chose a pragmatic approach that balances these concerns. Here's what that looks like in practice.

Training & Fine-tuning Pipeline

The implementation required careful attention to several technical details. Let's walk through the key components.

# Embedding-based similarity scoring
import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer('all-MiniLM-L6-v2')

def score_candidate(job_description: str, resume: str) -> dict:
    """Multi-field embedding comparison with bias handling."""
    job_emb = model.encode(job_description)
    resume_emb = model.encode(resume)

    # Cosine similarity
    similarity = np.dot(job_emb, resume_emb) / (
        np.linalg.norm(job_emb) * np.linalg.norm(resume_emb)
    )

    # Bias-aware scoring: reduce weight on demographic-correlated features
    adjusted_score = apply_bias_correction(similarity, resume)

    return {
        "raw_score": float(similarity),
        "adjusted_score": float(adjusted_score),
        "confidence": calculate_confidence(job_emb, resume_emb)
    }

This configuration reflects lessons learned from running similar setups in production. A few things to note:

1. Resource limits are essential — without them, a single misbehaving service can take down your entire stack. We learned this the hard way when a memory leak in one container consumed 14GB of RAM.

2. Volume mounts for persistence — never rely on container storage for data you care about. We mount everything to dedicated LVM volumes on SSD.

3. Health checks with real verification — a container being "up" doesn't mean it's "healthy." Always verify the actual service endpoint.

Common Pitfalls

We've seen teams make these mistakes repeatedly:

<div style="margin:2.5rem auto;max-width:600px;width:100%;text-align:center;"><svg viewBox="0 0 600 180" xmlns="http://www.w3.org/2000/svg" style="width:100%;height:auto;"><rect width="600" height="180" rx="12" fill="#1a1a2e"/><rect x="30" y="60" width="80" height="50" rx="25" fill="#3b82f6" opacity="0.85"/><text x="70" y="90" text-anchor="middle" fill="#ffffff" font-size="11" font-family="system-ui">Prompt</text><rect x="145" y="50" width="90" height="70" rx="8" fill="#6366f1" opacity="0.85"/><text x="190" y="80" text-anchor="middle" fill="#ffffff" font-size="10" font-family="system-ui">Embed</text><text x="190" y="95" text-anchor="middle" fill="#ffffff" font-size="10" font-family="system-ui">[0.2, 0.8...]</text><rect x="270" y="50" width="90" height="70" rx="8" fill="#a855f7" opacity="0.85"/><text x="315" y="75" text-anchor="middle" fill="#ffffff" font-size="10" font-family="system-ui">Vector</text><text x="315" y="90" text-anchor="middle" fill="#ffffff" font-size="10" font-family="system-ui">Search</text><text x="315" y="105" text-anchor="middle" fill="#ffffff" font-size="9" font-family="system-ui" opacity="0.7">top-k=5</text><rect x="395" y="50" width="90" height="70" rx="8" fill="#2dd4bf" opacity="0.85"/><text x="440" y="80" text-anchor="middle" fill="#1a1a2e" font-size="11" font-family="system-ui" font-weight="bold">LLM</text><text x="440" y="95" text-anchor="middle" fill="#1a1a2e" font-size="9" font-family="system-ui">+ context</text><rect x="520" y="60" width="55" height="50" rx="25" fill="#f59e0b" opacity="0.85"/><text x="547" y="90" text-anchor="middle" fill="#1a1a2e" font-size="10" font-family="system-ui">Reply</text><defs><marker id="arrow4" markerWidth="8" markerHeight="6" refX="8" refY="3" orient="auto"><path d="M0,0 L8,3 L0,6" fill="#e2e8f0"/></marker></defs><line x1="112" y1="85" x2="143" y2="85" stroke="#e2e8f0" stroke-width="1.5" marker-end="url(#arrow4)"/><line x1="237" y1="85" x2="268" y2="85" stroke="#e2e8f0" stroke-width="1.5" marker-end="url(#arrow4)"/><line x1="362" y1="85" x2="393" y2="85" stroke="#e2e8f0" stroke-width="1.5" marker-end="url(#arrow4)"/><line x1="487" y1="85" x2="518" y2="85" stroke="#e2e8f0" stroke-width="1.5" marker-end="url(#arrow4)"/><text x="300" y="155" text-anchor="middle" fill="#94a3b8" font-size="10" font-family="system-ui">Retrieval-Augmented Generation (RAG) Flow</text></svg><p style="margin-top:0.75rem;font-size:0.85rem;color:#94a3b8;font-style:italic;line-height:1.4;">RAG architecture: user prompts are embedded, matched against a vector store, then fed to an LLM with retrieved context.</p></div>

Production Deployment

In production, this approach has delivered measurable results:

|--------|--------|-------|-------------|

These numbers come from our actual production infrastructure running 90+ containers on a single server — proving that you don't need expensive cloud services to run reliable, scalable systems.

What We'd Do Differently

If we were starting today, we'd:

Monitoring & Iteration

Building scaling conversational ai: designing chat interfaces for financial services taught us several important lessons:

1. Start with the problem, not the technology — the best architecture is the one that solves your specific constraints 2. Measure everything — you can't improve what you don't measure 3. Automate the boring stuff — manual processes are error-prone and don't scale 4. Plan for failure — every system fails eventually; the question is how gracefully

If you're tackling a similar challenge, we've been there. We've shipped 36+ products across 8 industries, and we're happy to share our experience.

<div style="margin:2.5rem auto;max-width:600px;width:100%;text-align:center;"><svg viewBox="0 0 600 160" xmlns="http://www.w3.org/2000/svg" style="width:100%;height:auto;"><rect width="600" height="160" rx="12" fill="#1a1a2e"/><rect x="20" y="40" width="80" height="60" rx="6" fill="#3b82f6" opacity="0.85"/><text x="60" y="65" text-anchor="middle" fill="#ffffff" font-size="10" font-family="system-ui">Raw</text><text x="60" y="80" text-anchor="middle" fill="#ffffff" font-size="10" font-family="system-ui">Data</text><rect x="125" y="40" width="80" height="60" rx="6" fill="#6366f1" opacity="0.85"/><text x="165" y="65" text-anchor="middle" fill="#ffffff" font-size="10" font-family="system-ui">Pre-</text><text x="165" y="80" text-anchor="middle" fill="#ffffff" font-size="10" font-family="system-ui">process</text><rect x="230" y="40" width="80" height="60" rx="6" fill="#a855f7" opacity="0.85"/><text x="270" y="65" text-anchor="middle" fill="#ffffff" font-size="10" font-family="system-ui">Train</text><text x="270" y="80" text-anchor="middle" fill="#ffffff" font-size="10" font-family="system-ui">Model</text><rect x="335" y="40" width="80" height="60" rx="6" fill="#2dd4bf" opacity="0.85"/><text x="375" y="65" text-anchor="middle" fill="#1a1a2e" font-size="10" font-family="system-ui">Evaluate</text><text x="375" y="80" text-anchor="middle" fill="#1a1a2e" font-size="10" font-family="system-ui">Metrics</text><rect x="440" y="40" width="80" height="60" rx="6" fill="#f59e0b" opacity="0.85"/><text x="480" y="65" text-anchor="middle" fill="#1a1a2e" font-size="10" font-family="system-ui">Deploy</text><text x="480" y="80" text-anchor="middle" fill="#1a1a2e" font-size="10" font-family="system-ui">Model</text><rect x="545" y="40" width="40" height="60" rx="6" fill="#6366f1" opacity="0.6"/><text x="565" y="75" text-anchor="middle" fill="#ffffff" font-size="9" font-family="system-ui">Mon</text><defs><marker id="arrow3" markerWidth="8" markerHeight="6" refX="8" refY="3" orient="auto"><path d="M0,0 L8,3 L0,6" fill="#e2e8f0"/></marker></defs><line x1="102" y1="70" x2="123" y2="70" stroke="#e2e8f0" stroke-width="1.5" marker-end="url(#arrow3)"/><line x1="207" y1="70" x2="228" y2="70" stroke="#e2e8f0" stroke-width="1.5" marker-end="url(#arrow3)"/><line x1="312" y1="70" x2="333" y2="70" stroke="#e2e8f0" stroke-width="1.5" marker-end="url(#arrow3)"/><line x1="417" y1="70" x2="438" y2="70" stroke="#e2e8f0" stroke-width="1.5" marker-end="url(#arrow3)"/><line x1="522" y1="70" x2="543" y2="70" stroke="#e2e8f0" stroke-width="1.5" marker-end="url(#arrow3)"/><path d="M375,102 L375,130 L270,130 L270,102" stroke="#f59e0b" stroke-width="1" stroke-dasharray="4,3" fill="none" marker-end="url(#arrow3b)"/><defs><marker id="arrow3b" markerWidth="8" markerHeight="6" refX="8" refY="3" orient="auto-start-reverse"><path d="M0,0 L8,3 L0,6" fill="#f59e0b"/></marker></defs><text x="322" y="143" text-anchor="middle" fill="#f59e0b" font-size="9" font-family="system-ui">retrain loop</text></svg><p style="margin-top:0.75rem;font-size:0.85rem;color:#94a3b8;font-style:italic;line-height:1.4;">ML pipeline: from raw data collection through training, evaluation, deployment, and continuous monitoring.</p></div>

Ready to Build Something Similar?

We offer a unique deal: we'll build your demo for free. If you love it, we work together. If not, you walk away — no questions asked. That's how confident we are in our work.

*Tags: conversational AI financial services chatbot, BizStreet, ai-ml*