1. Anthropic Maps How LLMs Process Emotion

Anthropic published new interpretability research examining how emotional concepts are represented and utilized inside large language models. The work studies the internal processing systems that govern how models handle emotional representations — part of Anthropic’s broader program to understand what models are actually doing rather than what we assume they do.

Any team using system prompts to control tone — friendly for customer support, precise for technical documentation, professional for sales — is relying on the model’s internal representation of emotional and tonal concepts. Understanding how those representations actually work informs better prompt engineering and more targeted preference training. It also raises a design question: if models have internal structures that process emotional framing, then tone is not just a surface-level instruction but something the model reasons about at a deeper level. For teams running DPO or RLHF where tone is a preference criterion, this research offers a window into the mechanism being optimized.

2. Evo2 — A Foundation Model That Writes Chromosomes

Arc Institute released Evo2, a 300-billion-parameter genomics foundation model with a 131,000-token context window, trained on 9.3 trillion nucleotides from the OpenGenome2 dataset across multiple species. It can generate entire functional chromosomes and predict the effects of genetic mutations.

Previous DNA models were limited to roughly 8,000 tokens — far too short to capture the long-range dependencies that govern how genes actually function. Evo2’s leap to 131K context is what makes chromosome-scale generation possible. The release philosophy is equally significant: in a field where bio-AI models are increasingly locked behind pharmaceutical company walls, Arc Institute published both open weights and the full training dataset. The “scale plus long context” approach that transformed language models transfers directly to biological sequences. That finding extends well beyond genomics.

3. Open Models Cross the Threshold — and Start Healing Themselves

LangChain published two significant pieces this week. The first is an analysis showing that open models like GLM-5 and MiniMax M2.7 now match closed frontier models on core agent capabilities, while delivering significantly lower costs and faster latency in production. The second describes a self-healing deployment pipeline for production agents — an automated system that detects regressions after each deploy, diagnoses the root cause, and opens a pull request with a fix before a human needs to intervene.

Together, these two developments mark a shift in maturity. The capability gap between open and closed models has functionally closed for agentic tasks — the economics now favour self-hosted inference, where per-token API billing is replaced by fixed infrastructure costs. And the production story is catching up too. Building a capable agent is one challenge; keeping it working as dependencies shift and data distributions drift is another. The self-healing pattern closes the loop from detection to remediation automatically, transforming maintenance from a reactive burden into an automated feedback loop. Open models are not just matching closed ones on benchmarks — the tooling around them is reaching production grade.

4. Gemma 4 Arrives with MoE, Agentic Design, and a Ready Serving Stack

Google DeepMind released Gemma 4, their most capable open model family to date. The architecture combines Mixture of Experts with native multimodal input, structured reasoning, and tool-use capabilities. vLLM v0.19.0 shipped the same week with full Gemma 4 support, along with zero-bubble async scheduling and speculative decoding — 448 commits from 197 contributors.

MoE decouples capability from inference cost. A model with a large total parameter count but fewer active parameters per token delivers frontier-level quality at a fraction of the compute. The simultaneous vLLM release means the path from download to production serving is already clear. Speculative decoding, which generates candidate tokens with a smaller draft model and verifies them in batch, adds a meaningful latency improvement on top. For teams maintaining a model evaluation matrix, Gemma 4 is an immediate candidate with its serving infrastructure already in place.

5. HAIC — A New Framework for Evaluating AI in Teams

MIT Technology Review published a piece arguing that current AI evaluation methods are fundamentally misaligned with how AI is actually deployed. Benchmarks test isolated tasks against human performance, but production AI operates as part of collaborative human-AI teams within organizational workflows.

The proposed alternative is HAIC — Human-AI, Context-Specific Evaluation. Instead of asking “can this model solve this problem alone?”, HAIC asks “does this model improve outcomes when embedded in a team over time?” The distinction changes what gets optimized. Current benchmarks reward standalone capability. HAIC rewards integration quality, collaborative efficiency, and sustained performance under real-world conditions. For teams that have seen a model ace benchmarks and then underperform in production, this framework offers a concrete direction forward. Benchmarks drive development priorities — when the benchmarks measure the wrong thing, the priorities follow.

6. Holo3 — An Open Model for Computer Use

HCompany released Holo3, an open model for computer use and UI automation tasks. The model pushes the boundaries of how agents interact with desktop and web interfaces — clicking, typing, navigating, and completing multi-step workflows across applications.

Computer use is a rapidly evolving agent category. Anthropic’s Claude, OpenAI’s Operator, and Google’s Project Mariner have all staked positions, but these are closed systems. Holo3 is an open-weight alternative published directly on Hugging Face. For teams building agent-based products, an open computer-use model means the capability can be fine-tuned, self-hosted, and integrated into custom workflows without API dependencies. As agentic architectures move from chat-based interactions to tool-use and environment manipulation, computer use becomes a core capability layer rather than a novelty demo.

What Ties These Together

Open models are not just closing the capability gap with closed alternatives — they are building an ecosystem around themselves. Better serving infrastructure ships in lockstep with new architectures. Interpretability research reveals what these models are actually doing under the hood. Evaluation frameworks are evolving to match how AI is actually used. Production patterns are maturing from monitoring to automated remediation. The question has shifted from “can open models compete?” to “what does the full production stack look like when open models are the default?” That is where the industry is now building.

This isn’t inherently better or worse. It’s a different set of tradeoffs: swap SaaS convenience for control, predictability, and (usually) lower costs as you scale.

What Forgejo Provides

Forgejo started as a soft fork of Gitea in 2022 over governance concerns. In February 2024, it became a hard fork under Codeberg e.V., achieving full independence from its upstream. The latest stable release is v13.0 (October 2025), bringing WebAuthn support and 15% memory optimizations.

Core functionality:

• Git repository hosting

• Pull request and code review workflow

• Issue tracking with labels and milestones

• Built-in CI/CD (Forgejo Actions)

• Package registry (Docker, npm, Maven, PyPI)

• Git LFS support

• OAuth/LDAP/SAML authentication

• Federation for multi-instance collaboration

• AI code suggestions (opt-in plugins)

The interface resembles GitHub. Teams familiar with GitHub’s workflow navigate Forgejo without training.

Cost Structure

GitHub Teams: $5/user/month ($60/year per user) GitLab Premium: $32/user/month ($384/year per user)

For 10 users:

• GitHub: $600/year

• GitLab: $3,840/year

Forgejo on a VPS:

• Hetzner CX21: €4.85/month (~$52/year)

• Digital Ocean Droplet (2GB): $12/month ($144/year)

These VPS prices work for 10 users or 100 users. No per-seat scaling.

Hidden costs:

• Setup time: 4-6 hours

• Ongoing maintenance: 2-4 hours/month

• Backup storage: $1-5/month

Technical Requirements

Minimum specs:

• 1GB RAM (tight, but usable)

• 2 CPU cores

• 20GB storage + repository sizes

• Linux server (Ubuntu, Debian)

Recommended for production:

• 2GB RAM

• 2-4 CPU cores

• Storage: 50GB base + 1.3x repository sizes

• SSD strongly recommended

Database options:

• SQLite: Simple, single file. Works until concurrent operations slow down (10-20 active users).

• PostgreSQL: Better concurrency. Migrate when SQLite bottlenecks.

Installation

Docker Compose setup with health checks:

services:
  forgejo:
    image: codeberg.org/forgejo/forgejo:9-rootless
    container_name: forgejo
    environment:
      - USER_UID=1000
      - USER_GID=1000
      - FORGEJO__database__DB_TYPE=postgres
      - FORGEJO__database__HOST=db:5432
      - FORGEJO__database__NAME=forgejo
      - FORGEJO__database__USER=forgejo
      - FORGEJO__database__PASSWD=your_password_here
      - FORGEJO__server__ROOT_URL=https://git.yourdomain.com
      - FORGEJO__server__SSH_PORT=222
    volumes:
      - ./data:/data
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    ports:
      - "3000:3000"
      - "222:22"
    depends_on:
      db:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000"]
      interval: 30s
      timeout: 10s
      retries: 3
    restart: unless-stopped

  db:
    image: postgres:17-alpine
    environment:
      - POSTGRES_USER=forgejo
      - POSTGRES_PASSWORD=your_password_here
      - POSTGRES_DB=forgejo
    volumes:
      - ./postgres:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U forgejo"]
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped

Notes:

• Rootless image for simpler permissions

• SSH on port 222 avoids host conflicts

• Health checks ensure service readiness

• PostgreSQL 17 for latest performance improvements

Start: docker-compose up -d

First run launches web installer at

http://your-server:3000

. Configure admin credentials, server URL, and email settings.

Reverse Proxy and SSL

Forgejo serves HTTP on port 3000. Production needs HTTPS.

Caddy (automatic SSL):

git.yourdomain.com {
    reverse_proxy forgejo:3000
}

Nginx:

server {
    listen 443 ssl http2;
    server_name git.yourdomain.com;
    
    ssl_certificate /etc/letsencrypt/live/git.yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/git.yourdomain.com/privkey.pem;
    
    location / {
        proxy_pass http://localhost:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Get certificates: certbot --nginx -d git.yourdomain.com

Basic Usage

Repositories:

Organizations group repositories and manage team permissions. Create organizations for team projects, user accounts for personal repos.

Clone with custom SSH port:

git clone ssh://git@git.yourdomain.com:222/org/repo.git

Pull Requests:

Standard GitHub workflow:

1. Branch or fork

2. Push changes

3. Open pull request

4. Inline code review

5. Merge when approved

Issues:

Support Markdown, labels, milestones, assignees, attachments. Reference commits: fixes #123 creates automatic links.

Webhooks:

POST JSON to external endpoints on push, PR, issue, or release events. Configure in repository settings for CI/CD triggers.

Federation:

Connect multiple Forgejo instances for cross-instance collaboration. Users on one instance can interact with repositories on another. Enable in admin settings.

CI/CD: Forgejo Actions

GitHub Actions-compatible. Run your own runners.

Runner Setup:

services:
  runner:
    image: code.forgejo.org/forgejo/runner:3.3.0
    environment:
      - FORGEJO_INSTANCE_URL=https://git.yourdomain.com
      - FORGEJO_RUNNER_REGISTRATION_TOKEN=your_token
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    restart: unless-stopped

Token from: Site Administration → Actions → Runners

Workflow Example:

.forgejo/workflows/test.yml:

name: Test
on: [push, pull_request]

jobs:
  test:
    runs-on: docker
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v4
        with:
          python-version: '3.11'
      - run: pip install -r requirements.txt
      - run: pytest

Most GitHub Actions work without modification.

AI Code Suggestions

V13 introduces opt-in AI plugins for code completion and suggestions. Configure via admin panel. Integrates with OpenAI API or self-hosted models. Privacy-focused: code never leaves your infrastructure if using local models.

Enable in: Site Administration → AI Settings

Backup Strategy

Critical data:

1. Repositories (./data volume)

2. PostgreSQL database

3. Configuration (app.ini)

Daily Backup:

#!/bin/bash
BACKUP_DIR="/backups/forgejo"
DATE=$(date +%Y%m%d_%H%M%S)

# Database backup (no downtime)
docker-compose exec -T db pg_dump -U forgejo forgejo | gzip > $BACKUP_DIR/db_$DATE.sql.gz

# Repository backup
tar -czf $BACKUP_DIR/data_$DATE.tar.gz ./data

# Retention: 7 days
find $BACKUP_DIR -mtime +7 -delete

Off-site:

rclone sync /backups/forgejo s3:bucket-name/forgejo

Test restores regularly.

Updates

Updates every 4-8 weeks.

docker-compose pull
docker-compose up -d
docker-compose logs -f forgejo

Downtime: 30-60 seconds.

Security:

• Enable 2FA for admins

• SSH key-based auth only

• Strong database passwords

• Keep host OS updated

When to Self-Host

Good fit:

• You manage servers already

• Stable team size

• Data sovereignty requirements

• Infrastructure control preference

Poor fit:

• No operational expertise

• Rapid unpredictable growth

• Critical uptime without HA capability

• Small teams focused on product

Getting Started

Evaluation:

1. Deploy local instance

2. Create test repository

3. Try workflows (clone, commit, PR)

4. Test Actions

5. Practice backup/restore

Resources:

• Docs: forgejo.org/docs

• Forum: codeberg.org/forgejo/discussions

• Matrix: #forgejo:matrix.org

• Video: Forgejo Tutorial by Awesome Open Source

Bottom Line

Forgejo trades SaaS convenience for infrastructure control. You eliminate per-seat costs and external dependencies at the cost of operational responsibility.

The software is stable. Choose based on whether managing it aligns with your team’s capabilities, not ideology.

The problem isn’t carelessness. It’s that PII (Personally Identifiable Information) is often invisible until you know to look for it.

Subscribe now

What Counts as PII?

PII is any data that can identify a specific individual, either directly or when combined with other information. The definition varies by regulation, but generally includes:

Direct identifiers — Data that points to a specific person on its own:

• Full names

• Email addresses

• Phone numbers

• Social Security Numbers

• Passport and driver’s license numbers

• Biometric data

Indirect identifiers — Data that can identify someone when combined:

• IP addresses

• Device IDs

• Location data

• Dates of birth

• Employment information

Financial data — Often regulated separately but equally sensitive:

• Credit card numbers

• Bank account and routing numbers

• IBAN codes

Authentication secrets — Not traditionally “PII” but equally dangerous:

• API keys and tokens

• Passwords

• Private keys

• Session tokens

The last category is often overlooked. An exposed AWS key isn’t personal information, but it can grant access to systems containing millions of personal records. The blast radius of a leaked credential often exceeds that of a leaked SSN.

Credentials are the keys to the PII vault. A single exposed API key can unlock databases containing millions of personal records. That’s why effective scanning must detect both PII and secrets.

Where PII Hides

The obvious places—databases, CRM systems, HR files—usually have controls. The risk is in the unstructured data that flows through daily work:

Support tickets: A customer reports a bug and includes their full account details. The ticket gets escalated, exported to a spreadsheet, shared with engineering. Each hop increases exposure.

Log files: Application logs capture request parameters, user IDs, IP addresses, sometimes full payloads. Developers copy these into debugging sessions, paste them into chat, attach them to tickets.

Code repositories: Test files contain sample data. Configuration files contain connection strings. Comments contain “temporary” credentials. README files contain example API calls with real tokens.

AI prompts: Users paste customer conversations, error messages, database queries into ChatGPT or Claude for help. These prompts may be used for model training unless explicitly opted out.

Email threads: A message gets forwarded, then forwarded again. By the fifth hop, nobody remembers that the original contained a customer’s SSN in the signature block.

Screenshots: A developer shares a screenshot of a bug. The browser’s address bar shows a URL with a session token. The page content shows a user’s profile.

The Regulatory Landscape

Data protection regulations have teeth. Under GDPR, fines can reach €20 million or 4% of global annual revenue—whichever is higher. CCPA allows statutory damages of $100–$750 per consumer, per incident. A single leak of 1,000 customer emails could theoretically result in a $750,000 liability.

But the real cost is often operational:

• Breach notification requirements: GDPR requires notification within 72 hours. This means incident response, legal review, customer communication—all on a tight timeline.

• Right to erasure: If you can’t track where data has been copied, you can’t guarantee deletion.

• Audit requirements: Demonstrating compliance requires knowing what data you have and where it lives.

Most breaches don’t make headlines. They’re discovered during audits, reported by customers, or found by security researchers. The exposure may have existed for months before detection.

Detection is Harder Than It Looks

Why doesn’t everyone just scan for PII before sharing? Because detection is genuinely difficult:

Format variation: Phone numbers appear as (555) 123-4567, 555-123-4567, +1 555 123 4567, 5551234567. Email addresses get obfuscated as john[at]example[dot]com. Credit cards have spaces, dashes, or neither.

False positives: A 9-digit number might be an SSN or a random ID. A 16-digit number might be a credit card or a tracking number. Without validation, scanners either miss things or flag everything.

Context matters: “John Smith” in a novel isn’t PII. “John Smith, Account #12345” in a support ticket is. Simple pattern matching can’t distinguish.

Secrets are diverse: AWS keys start with AKIA. GitHub tokens start with ghp_. Stripe keys start with sk_live_. Generic API keys follow no pattern at all. Each requires specific detection logic.

Encoding layers: Data gets base64 encoded, embedded in JSON, nested in XML. A scanner that only checks surface text misses encoded content.

Building a Detection Approach

Effective PII detection combines multiple techniques:

Pattern matching handles well-formatted data. SSNs follow XXX-XX-XXXX. Credit cards match specific prefixes (4 for Visa, 5 for Mastercard). Email addresses have predictable structure.

Checksum validation reduces false positives. Credit card numbers include a check digit validated by the Luhn algorithm. IBANs have country-specific formats with built-in verification. A random 16-digit number fails these checks.

Prefix detection catches credentials. Cloud provider keys use identifiable prefixes: AKIA for AWS access keys, ghp_ for GitHub tokens, sk_live_ for Stripe, AIza for Google APIs. These prefixes exist specifically to enable detection.

Confidence scoring acknowledges uncertainty. A pattern match against an SSN format with proper separators is high confidence. A 9-digit number without context is low confidence. Surfacing the score lets users prioritize review.

A Tool to Help

We built Privacy Scanner to make PII detection accessible. It’s a free, browser-based tool that identifies sensitive data in text and files.

The scanner detects:

• Email addresses (including obfuscated formats)

• Phone numbers (US and international)

• Social Security Numbers

• Credit cards (with Luhn validation)

• Physical addresses (US, UK, EU formats)

• Bank account numbers and IBANs

• Cloud credentials (AWS, GitHub, Stripe, Google, Azure, Slack)

• JWT tokens and private key headers

• Passwords in plaintext

Each detection includes a confidence score and contributes to an overall risk assessment. The tool generates a redacted preview you can copy directly.

For sensitive use cases, there’s a “Browsers only mode” where your text never leaves the browser—the backend only returns coordinates, and masking happens locally.

No signup required. No data stored.

Try it here →

Building Habits

Tools help, but habits matter more. Some practices that reduce accidental exposure:

Assume logs contain PII. Before sharing any log output, scan it. Better yet, configure your logging framework to redact sensitive fields at the source.

Sanitize before escalation. When forwarding a customer issue, take 30 seconds to remove identifying details that aren’t necessary for resolution.

Use separate test data. Maintain a library of fake but realistic test data. Never copy production data into development environments without anonymization.

Review before commit. Add PII scanning to your pre-commit hooks. Catch credentials and test data before they enter version control.

Question AI prompts. Before pasting into an LLM, ask: does this contain customer data? Could this identify someone? Is there a way to get the same help with anonymized input?

Privacy incidents rarely involve sophisticated attacks. They happen when ordinary people do ordinary things without realizing what’s embedded in the data they’re handling. The fix isn’t perfect security—it’s awareness and accessible tools.

Questions or feedback? Post your comments, we're improving the scanner based on feedback.

Why Compare vLLM with HuggingFace Transformers?

HuggingFace Transformers is the de facto standard library for working with large language models. It provides an accessible, well-documented interface that most ML practitioners use for training, fine-tuning, and inference. The library has facilitated access to state-of-the-art models and serves as the foundation for countless AI applications.

However, HuggingFace was designed primarily as a research and development tool, not as a production inference engine optimized for serving real users at scale. When deploying LLMs in production—especially for applications serving multiple concurrent users—HuggingFace’s straightforward approach becomes a significant bottleneck.

• Each request is processed sequentially

• GPU memory is allocated statically regardless of actual usage

• There’s no optimization for common patterns like shared system prompts.

These limitations compound quickly: a system that feels responsive during development becomes frustratingly slow under real-world load. This is precisely where vLLM enters the picture. vLLM is purpose-built for high-throughput LLM serving in production environments. It introduces several architectural innovations that fundamentally change how inference is performed:

• PagedAttention for efficient memory management,

• Continuous batching for handling concurrent requests dynamically, and

• Prefix caching for scenarios with repeated prompt patterns.

The benchmark validates whether these claimed improvements translate to real-world performance gains using a fine-tuned Qwen 2.5 7B model. The following 5 tests were conducted :

[1] Throughput Comparison (Sequential vs Batched vs vLLM)

This is the fundamental comparison that answers the core question: how much faster is vLLM? We measured how many tokens per second each approach can generate when processing 20 identical prompts. Throughput directly translates to cost and user experience. Higher throughput means serving more users with the same hardware, reducing infrastructure costs, and delivering faster responses. In a production environment, throughput determines how many concurrent users your system can handle before response times become unacceptable.

The results are striking and validate vLLM’s core value proposition. HuggingFace sequential processing—the default approach most developers use when following tutorials and documentation—achieved only 40.9 tokens per second. vLLM delivered 868.5 tokens per second, a 21.2x improvement. Even compared to HuggingFace’s batched inference (which requires additional code complexity and careful tuning), vLLM was 1.46x faster while being simpler to deploy and configure.

[2] Maximum Concurrent Sequences (KV Cache Efficiency)

This test pushed the system to handle increasing numbers of simultaneous requests (1, 10, 25, 50, 100, 150, 200) to identify the breaking point and understand scaling behaviour. Real-world applications don’t serve one user at a time. A local business assistant might hundreds of users querying simultaneously during peak hours. The ability to handle concurrent requests without crashing, running out of memory, or degrading severely determines whether the system is genuinely production-ready or merely a demo.

vLLM successfully handled all 200 concurrent sequences without failure. More importantly, throughput increased with concurrency—a counterintuitive result that demonstrates vLLM’s efficient batching strategy. The system achieved peak throughput of 3,184 tokens per second at maximum concurrency, nearly 70x faster than single-request HuggingFace performance. This remarkable scaling is possible because of PagedAttention, vLLM’s innovative memory management system. Traditional transformers allocate fixed memory blocks for each sequence’s key-value cache, leading to fragmentation and waste when sequences vary in length. PagedAttention allocates memory in small pages on-demand, similar to how operating systems manage virtual memory. This eliminates waste and allows significantly more sequences to run concurrently within the same GPU memory budget.

[3] Prefix Caching Impact

This test compared performance with and without prefix caching enabled, using 100 queries that shared a common system prompt—a scenario typical for chatbot and assistant applications. Production LLM applications typically use system prompts—detailed instructions that define the assistant’s behavior, personality, and constraints. For the fine-tuned model, every single query includes context about being a “helpful AI assistant for local business discovery ” that understands Hindi, English, Marathi, and Hinglish. Processing this identical prefix repeatedly for every user query wastes substantial computation.

Enabling prefix caching delivered a 32% speedup with zero code changes required. For the Discovery model’s specific use case—where every query shares the same system prompt defining its locality-focused, multilingual behavior—this optimization directly reduces latency and operational cost. Over thousands of daily queries, this improvement compounds into significant infrastructure savings and noticeably snappier user experiences.

[4] Concurrency Scaling

This test systematically measured how throughput and latency change as concurrent requests increase from 1 to 100. Understanding the scaling curve helps with capacity planning and setting realistic expectations. Does performance degrade gracefully under load, or does it collapse catastrophically? At what point do you need additional hardware?

The results demonstrate excellent scaling characteristics that defy naive expectations. Throughput increased nearly linearly with concurrency up to about 50 users, then continued improving at a diminishing but still positive rate. Remarkably, per-request latency dropped from 2.8 seconds (single request) to an effective 42ms per request when processing 100 concurrent requests in 4.2 seconds total.

This counterintuitive behavior demonstrates continuous batching working exactly as designed: the GPU stays fully utilized processing multiple requests simultaneously, and individual requests benefit from efficient batched matrix operations.

[5] Chunked Prefill

This test evaluated chunked prefill performance with long-context prompts containing approximately 1,128 tokens each. Long prompts can cause latency spikes as the system must process the entire input before generating any output tokens. Chunked prefill addresses this by breaking prefill computation into smaller pieces, reducing time-to-first-token and improving perceived responsiveness.

For this moderate context length, chunked prefill showed minimal difference (~1%). This optimization becomes substantially more important with very long contexts exceeding 4,000 tokens. For typical Discovery model queries, which involve relatively short user questions, this feature is less critical but remains available when processing longer documents or conversation histories.

Conclusion

The benchmark comprehensively validates vLLM’s performance claim against HuggingFace Transformers.

For the fine-tuned Qwen2.5-7B model specifically, these results demonstrate that a single A5000 GPU can serve a substantial user base effectively. The prefix caching optimization aligns perfectly with the model’s architectural use case, where every query shares the same system prompt defining its local business assistant persona.

The choice to benchmark against HuggingFace Transformers provides a meaningful baseline that practitioners universally understand. vLLM isn’t incrementally better—it’s an order of magnitude faster, representing the difference between a functional prototype and a genuinely production-ready system capable of serving real users reliably.

What is Exploratory Data Analysis?

Exploratory Data Analysis is an approach for examining datasets and to summarize their main characteristics, often using visual methods. Pioneered by statistician John Tukey in the 1970s, EDA emphasizes the importance of looking at data before making assumptions or building models. Unlike confirmatory analysis, which tests specific hypotheses, EDA is an open-ended process of discovery.

The primary objectives of EDA include:

• Understanding the structure and distribution of data

• Detecting outliers and anomalies

• Identifying patterns, trends, and relationships between variables

• Testing underlying assumptions

• Selecting appropriate models for further analysis

EDA is not a rigid set of procedures but rather a state of mind that encourages curiosity and thorough investigation. It asks questions like: What does the data tell us? What doesn’t it tell us? What patterns emerge? What surprises exist?

The Power of Data Visualization in EDA

Data visualization transforms abstract numbers into concrete visual representations, making complex data accessible and understandable. Our brains process visual information a lot faster than the text, making visualization an indispensable tool for EDA. Well-designed visualizations can reveal trends, outliers, and relationships that might take hours to discover through statistical analysis alone.

Key visualization techniques in EDA include:

Histograms and Distribution Plots: Reveal the shape and spread of data

Scatter Plots: Uncover relationships between variables

Box Plots: Identify outliers and compare distributions across groups

Correlation Heatmaps: Display relationships between multiple variables

Time Series Plots: Track changes over time

Animated Visualizations: Show evolution of patterns across dimensions

Understanding the Gapminder Dataset

The Gapminder dataset, curated by the Gapminder Foundation, provides a fascinating lens through which to explore global development trends. It contains data for 142 countries from 1952 to 2007, tracking three critical metrics:

Life Expectancy

This metric measures the average number of years a person is expected to live at birth. In our dataset, life expectancy ranges from a sobering 23.6 years (Rwanda, 1992, during the genocide) to an impressive 82.6 years (Japan, 2007). This dramatic range immediately tells a story about global health disparities and the impact of conflict on human welfare.

Population

Population figures range from 60,011 (Sao Tome and Principe) to over 1.3 billion (China, 2007). This metric helps us understand demographic pressures, economic potential, and resource allocation challenges facing different nations.

GDP per Capita

Measured in international dollars, GDP per capita ranges from $241.17 to $113,523.13 (Kuwait during an oil boom). This economic indicator reveals the vast wealth inequality between nations and provides context for understanding quality of life differences.

Key Statistical Metrics in EDA

Measures of Central Tendency

• Mean : The average provides a quick snapshot but can be skewed by outliers

• Median : The middle value is more robust to extreme values

• Mode : The most frequent value, particularly useful for categorical data

For the Gapminder dataset, comparing mean and median GDP per capita reveals positive skewness—a few wealthy nations pull the average higher than the median, indicating wealth concentration.

Measures of Dispersion

• Standard Deviation : Quantifies variation around the mean

• Range : The difference between maximum and minimum values

• Interquartile Range (IQR) : The spread of the middle 50% of data, robust to outliers

• Coefficient of Variation (CV) : Enables comparison of variability across different scales

The CV is particularly valuable when comparing metrics with different units. For instance, while population has enormous absolute variation, its CV reveals whether this variation is proportionally larger than life expectancy’s variation.

Percentiles and Quartiles

Percentiles divide data into 100 equal parts, providing detailed distribution information. The 25th, 50th (median), and 75th percentiles are especially important, forming the basis of box plots and revealing data symmetry or skewness.

Correlation Analysis

Correlation coefficients (ranging from -1 to +1) measure the strength and direction of linear relationships between variables. In the Gapminder data, life expectancy and GDP per capita show strong positive correlation (typically > 0.7), suggesting that wealthier nations tend to have longer-lived populations. However, it’s crucial to remember that correlation does not imply causation.

Advantages of Systematic EDA

Early Problem Detection

EDA reveals data quality issues before they corrupt analysis results. Missing values, incorrect data types, duplicate records, and inconsistent formatting become apparent through summary statistics and visualizations.

Insight Generation

Unexpected patterns often emerge during EDA. For example, examining the Gapminder data might reveal that some countries experienced dramatic life expectancy drops during specific years, prompting investigation into historical events like wars or epidemics.

Communication Enhancement

Visualizations transcend language barriers and technical expertise levels. A well-designed animated scatter plot showing the evolution of life expectancy versus GDP over time can convey decades of development history in seconds, making data accessible to stakeholders at all levels.

Model Selection Guidance

Understanding data distributions, relationships, and outliers helps select appropriate analytical methods. For instance, discovering non-linear relationships between variables suggests the need for polynomial regression rather than simple linear models.

Common Pitfalls in Data Analysis

Confirmation Bias

Analysts sometimes unconsciously seek patterns that confirm pre-existing beliefs while ignoring contradictory evidence. EDA should be approached with an open mind, allowing data to speak rather than forcing it into predetermined narratives.

Over-reliance on Means

The mean can be misleading when data is skewed or contains outliers. Always examine medians and full distributions. For GDP data, the mean might suggest average prosperity while hiding extreme inequality.

Ignoring Context

Statistical patterns without domain knowledge can lead to absurd conclusions. A correlation between ice cream sales and drowning deaths doesn’t mean ice cream causes drowning—both increase during summer.

Misleading Visualizations

Poor chart choices, manipulated axes, cherry-picked data ranges, and inappropriate color schemes can distort reality. Always ensure visualizations accurately represent data without exaggeration or omission.

Correlation-Causation Confusion

Perhaps the most dangerous pitfall is inferring causation from correlation. While Gapminder shows strong correlation between GDP and life expectancy, this doesn’t prove that increasing GDP directly causes longer lives. Confounding variables like education, healthcare access, and sanitation play crucial roles.

Neglecting Outliers

While outliers can distort analyses, they often contain valuable information. Rwanda’s drastically low life expectancy in 1992 isn’t noise to be removed—it’s a crucial data point reflecting historical tragedy that demands acknowledgment and investigation.

Analysis Paralysis

EDA is meant to be iterative and exploratory, but endless exploration without moving toward conclusions wastes resources. Setting clear objectives and timelines prevents perpetual analysis without action.

Conclusion

Exploratory Data Analysis, powered by thoughtful visualization, transforms raw data into actionable insights. The Gapminder dataset exemplifies how systematic exploration of metrics like life expectancy, population, and GDP per capita can reveal global development patterns, inequalities, and trends. By understanding key statistical measures—from means and medians to correlations and percentiles—and avoiding common analytical pitfalls, data analysts can extract genuine value from information.

In our data-driven world, EDA skills are not merely technical competencies but essential literacies for making informed decisions. Whether examining global health trends, business performance, or scientific phenomena, the principles remain constant: approach data with curiosity, visualize thoughtfully, measure carefully, and always question assumptions. Through disciplined EDA, we transform data from mere numbers into knowledge that drives understanding and progress.

Subscribe now

Subscribe now

This post draws from the Fast Prototyping of GenAI Apps with Streamlit course by DeepLearning.AI and Snowflake . This course is taught by Chanin Nantasenamat , Sr Developer Advocate also known as the Data Professor.

Why Fast Prototyping Matters

Ideas often lose momentum when they stay in documents or slide decks. In AI, where experimentation is key, the faster you can test and share, the better. Rapid prototyping helps you:

• Validate assumptions: Instead of debating whether an idea works, you can show it.

• Gather feedback early: A working demo sparks more useful conversations than abstract descriptions.

• Iterate quickly: You can refine based on real interactions, not speculation.

• Influence decisions: Stakeholders respond to tangible prototypes more than theoretical plans.

Generative AI adds another layer: with large language models (LLMs), you can build functional prototypes with minimal code. Streamlit provides the interface to make those prototypes usable and shareable.

What is Streamlit?

Streamlit is a Python library that transforms scripts into interactive web apps. Its appeal lies in simplicity:

• Minimal code: A few lines can create buttons, sliders, text inputs, or file uploaders.

• No front-end skills required: You don’t need to know HTML, CSS, or JavaScript.

• Instant sharing: Apps can be deployed on Streamlit Community Cloud or integrated into platforms like Snowflake.

For data scientists, researchers, and AI developers, this means you can focus on logic and models while still delivering polished, interactive demos.

The Prototyping Workflow

The course outlines a practical workflow for building GenAI apps with Streamlit. Here’s a simplified version:

1. Start Small
   Begin with a minimal app—often a chatbot powered by an LLM. The goal is to get something working quickly, not to perfect it.

2. Layer in Prompt Engineering
   Improve the quality of responses by refining prompts. Streamlit makes it easy to expose prompt variations through text boxes or dropdowns, so you can experiment interactively.

3. Add Retrieval-Augmented Generation (RAG)
   Connect your app to external data sources. For example, you might let the chatbot answer questions based on a company’s knowledge base or a dataset stored in Snowflake.

4. Deploy for Feedback
   Push the prototype to Streamlit Community Cloud or Snowflake. Share the link with colleagues or users and gather feedback.

5. Iterate
   Use the feedback to refine prompts, improve data connections, or adjust the interface. Because Streamlit apps are lightweight, iteration cycles are fast.

Example: A Simple Chatbot

Here’s a minimal Streamlit app that connects to an LLM (using OpenAI as an example):

import streamlit as st
import openai

# Set your OpenAI API key
openai.api_key = st.secrets[”OPENAI_API_KEY”]

st.title(”Quick Chatbot Prototype”)

# Initialize chat history
if “messages” not in st.session_state:
    st.session_state.messages = []

user_input = st.text_input(”Ask me anything:”)

if user_input:
    # Append user message to chat history
    st.session_state.messages.append({”role”: “user”, “content”: user_input})

    # Call OpenAI ChatCompletion with the full conversation history to maintain context
    response = openai.ChatCompletion.create(
        model=”gpt-3.5-turbo”,
        messages=st.session_state.messages
    )
    
    answer = response[”choices”][0][”message”][”content”]
    
    # Append assistant’s response to chat history
    st.session_state.messages.append({”role”: “assistant”, “content”: answer})
    
    # Display the assistant’s response
    st.write(answer)

This script:

• Creates a text input box.

• Sends the input to an LLM.

• Displays the response.

It’s only a few lines of code, but it produces a working chatbot you can share.

Adding Prompt Engineering

Prompt engineering is about shaping the model’s behavior. With Streamlit, you can expose prompt templates as editable fields:

import streamlit as st
import openai

# Set OpenAI API key securely
openai.api_key = st.secrets[”OPENAI_API_KEY”]

prompt_template = st.text_area(
    “Prompt template:”,
    “You are a helpful assistant. Answer clearly and concisely.\n\nUser: {question}\nAssistant:”
)

user_input = st.text_input(”Ask me anything:”)

if user_input:
    prompt = prompt_template.format(question=user_input)
    response = openai.Completion.create(
        model=”text-davinci-003”,
        prompt=prompt,
        max_tokens=200
    )
    st.write(response[”choices”][0][”text”])

Now you can experiment with different instructions without changing the code.

Adding RAG (Retrieval-Augmented Generation)

RAG combines LLMs with external data. For example, you might let the chatbot answer based on a set of documents. A simplified version looks like this:

import streamlit as st
import openai

openai.api_key = st.secrets[”OPENAI_API_KEY”]

prompt_template = st.text_area(
    “Prompt template:”,
    “You are a helpful assistant. Answer clearly and concisely.\n\nUser: {question}\nAssistant:”
)

user_input = st.text_input(”Ask me anything:”)

if user_input:
    prompt = prompt_template.format(question=user_input)
    response = openai.Completion.create(
        model=”text-davinci-003”,
        prompt=prompt,
        max_tokens=200
    )
    st.write(response[”choices”][0][”text”])

This setup allows the chatbot to ground its answers in your own dataset, making it more useful for specific domains.

Deployment

When building a prototype, keep things simple and focus on the main interaction—don’t add unnecessary features. Make it easy for users to adjust prompts, settings, or data inputs. Share your work early and improve it quickly through feedback. If your prototype handles sensitive data, make sure it runs in a secure environment.

Once your prototype works locally, you can deploy it:

• Streamlit Community Cloud: Free and simple for sharing demos.

• Snowflake + Streamlit: For secure, production-ready environments with enterprise data.

Deployment is as simple as pushing your code to GitHub and linking it to Streamlit Cloud.

Wrapping Up

Streamlit doesn’t replace production systems, but it gives you a powerful way to explore, test, and communicate ideas. In the fast-moving world of generative AI, that speed of exploration is often the difference between leading and lagging.

Rapid prototyping with Streamlit is less about building polished products and more about accelerating learning. By lowering the barrier to creating interactive GenAI apps, it allows individuals and teams to validate ideas quickly, gather feedback, and refine direction.

In practice, this means fewer stalled discussions and more tangible progress. Whether you’re experimenting with a chatbot, a summarizer, or a data assistant, the workflow is the same: start small, iterate fast, and share early.

Share ValueCurve

Subscribe now

[1] Samsung SAIL Montreal’s Tiny Recursive Models (TRM) represent a minimalist approach to AI reasoning, using a small neural network with just 7 million parameters to tackle complex tasks. TRM challenges the reliance on large language models by showing that recursive reasoning—where the model iteratively refines its answers—can yield strong results with minimal compute. The model begins with an embedded question, answer, and latent state, then updates its latent state and answer over multiple steps to improve accuracy.

TRM achieved notable scores on ARC-AGI benchmarks (45% on ARC-AGI-1 and 8% on ARC-AGI-2), levels typically reached by much larger models. It avoids complex theoretical constructs, focusing instead on practical recursion. The codebase is open-source, built in Python with CUDA and PyTorch, and has been tested on datasets like ARC-AGI, Sudoku-Extreme, and Maze-Hard. This work underscores the potential of compact models in AI safety and reasoning research.

[2] The Claude Agent SDK is a developer toolkit for building and deploying custom AI agents. It supports both TypeScript (for Node.js and web apps) and Python (for data science), with streaming and single input modes. The SDK is built on the Claude Code agent harness, offering prompt caching and performance optimization. Key features include automatic context management, error handling, session control, and monitoring—essential for production use. Agents can use built-in tools for file operations, code execution, and web search, and connect to external services via the Model Context Protocol (MCP).

Developers can define agent roles using System Prompts and control tool access with allowedTools or disallowedTools. The SDK supports Claude Code features like Subagents, Hooks, and Slash Commands through file-based configuration. It enables various agent types, including coding agents (e.g., SRE bots, code reviewers) and business agents (e.g., legal assistants, finance advisors, support bots). Authentication requires an API key via the ANTHROPIC_API_KEY environment variable, with optional support for Amazon Bedrock and Google Vertex AI. Overall, the SDK provides a structured, extensible foundation for building reliable, task-specific AI agents.

[3] Petri (Parallel Exploration Tool for Risky Interactions) is an open-source framework for auditing AI models by automating behavior testing across diverse scenarios. Built on the UK AI Security Institute’s Inspect framework, Petri supports most model APIs and reduces the manual effort needed for alignment evaluations. Its process includes four steps: forming hypotheses about risky behaviors, writing seed instructions for audit scenarios, running automated assessments via an auditor agent and a judge, and iterating based on transcript scores. The auditor agent simulates interactions with the target model, adjusting its approach dynamically. The judge scores transcripts across multiple dimensions, extracting highlights and summaries to identify misaligned behaviors.

Petri has surfaced issues like deception, oversight subversion, and whistleblowing in frontier models. In pilot tests, Claude Sonnet 4.5 and GPT-5 showed strong safety profiles, while others like Gemini 2.5 Pro and Grok-4 raised concerns. Limitations include realism gaps in transcripts, reliance on human-generated hypotheses, auditor model constraints, and judge subjectivity. Petri is extensible and includes 111 sample seed instructions, enabling rapid exploration and customization of audit tools and scoring systems.

[4] AgentKit is OpenAI’s full-stack platform for building, deploying, and optimizing AI agents, replacing earlier tools like the Agents SDK and Responses API. It includes Agent Builder, a visual canvas for designing multi-agent workflows with drag-and-drop nodes, preview runs, tool integration, and version control. ChatKit enables seamless embedding of chat-based agents into products or websites, handling streaming, thread management, and customizable UI. The Connector Registry provides enterprises with a centralized panel to manage data and tool integrations, including pre-built connectors and third-party Model Context Protocols (MCPs). Guardrails offer a modular safety layer to detect jailbreaks and protect sensitive data.

AgentKit also expands evaluation capabilities through Evals, supporting dataset creation, trace grading, prompt optimization, and third-party model assessment. For advanced tuning, it includes Reinforcement Fine-Tuning (RFT), available on o4-mini and in beta for GPT-5, allowing custom tool call training and grader configuration. As of October 2025, ChatKit and Evals are generally available, while Agent Builder remains in beta. AgentKit is designed to streamline agent development for both individual developers and enterprise teams.

[5] IBM Granite 4.0 is a family of open-source language models available on the Docker Hub model catalog, enabling developers to quickly build generative AI applications using Docker Model Runner. Designed for speed, flexibility, and cost-efficiency, Granite 4.0 combines enterprise-grade performance with a lightweight footprint, making it ideal for local prototyping and scalable deployment. Licensed under Apache 2.0, the models are customizable and commercially usable.

Technically, Granite 4.0 uses a hybrid architecture that merges Mamba-2’s linear efficiency with transformer precision, and select models apply a Mixture of Experts (MoE) strategy to reduce memory usage by over 70%. It also supports extremely long context lengths—up to 128,000 tokens—limited only by hardware. The model lineup includes H-Small (32B total, ~9B active) for RAG and agents on L4 GPUs, H-Tiny (7B total, ~1B active) for edge deployment on RTX 3060, H-Micro and Micro (3B dense) for ultra-light or fallback use cases. These variants support development on accessible hardware. With Docker Model Runner, developers can deploy models via an OpenAI-compatible API for tasks like document analysis, advanced RAG systems, multi-agent workflows, and edge AI applications.

[6] The ASUS Ascent GX10 is a compact desktop AI supercomputer built on NVIDIA DGX™ Spark and powered by the NVIDIA® GB10 Grace Blackwell Superchip. It delivers 1 petaFLOP of AI performance using FP4 and features a fifth-generation Blackwell GPU, 128 GB of LPDDR5x unified memory, and a high-performance 20-core Arm CPU for fast training and inference. With NVIDIA® NVLink™-C2C and ConnectX-7 networking, it supports scalable multi-GX10 setups for handling models like Llama 3.1 with 405 billion parameters. Designed for minimal footprint and high reliability, the GX10 includes QuietFlow Cooling, dual vapor chambers, and passes MIL-STD 810H durability tests. It supports up to five 4K displays and NVIDIA DLSS 4 for enhanced visuals.

The system runs NVIDIA DGX™ OS with Ubuntu and comes preloaded with CUDA, PyTorch, TensorFlow, Jupyter, TensorRT, NIM™, and Blueprints. It enables development and fine-tuning of models up to 200 billion parameters and supports workloads across generative AI, computer vision, analytics, and simulation. Models can be transitioned to DGX Cloud or other infrastructures with minimal code changes. Connectivity includes multiple USB-C ports, HDMI 2.1b, 10 GbE LAN, and a ConnectX-7 NIC, making it a powerful, developer-optimized platform for AI experimentation and deployment.

Together, these platforms reflect a shift toward modular, efficient, and developer-accessible AI infrastructure. From minimalist reasoning models and scalable agent frameworks to open-source language models and compact supercomputers, the ecosystem is evolving to support rapid prototyping, safe deployment, and high-performance experimentation across diverse AI workloads. These innovations empower researchers, developers, and enterprises to build more capable, aligned, and accessible AI systems.

Share ValueCurve

Subscribe now

The purpose of this post is to provide readers with a mental map for any ML project. Let’s dive in.

1. Data Collection

The process begins with gathering relevant data from various sources. These might include databases, logs, CSV files, APIs, and sensor feeds. It’s crucial to define what is needed before pulling in every field. Objectives and data schema should be sketched out early to save time in later stages. Privacy rules and security policies must be respected during record ingestion. Each dataset should be tagged with metadata that captures when, where, and how it arrived. This context supports traceability and reproducibility in analysis and modeling. A solid foundation of clean, well-understood inputs reduces surprises down the road.

2. Feature Engineering

Raw attributes rarely arrive in the shape and scale a model requires. Feature engineering transforms these inputs into meaningful variables. Numeric fields might be normalized, categorical ones encoded, or parts of timestamps extracted. Domain expertise helps craft interaction terms or rolling averages for time-series data. For text tasks, tokenization or embeddings convert words into numbers. Techniques like PCA or feature selection help manage high dimensionality. Each crafted feature should carry a clear rationale about its role in prediction. Well-engineered features often boost performance more than complex algorithms.

3. Model Training

With features in place, a model is trained to learn underlying patterns. The process starts by selecting an algorithm family—linear models, decision trees, or neural networks—based on the problem. Data is split into training and validation sets to catch overfitting before it sneaks in. Hyperparameter tuning is automated with grid search, random search, or Bayesian methods. Training metrics like loss curves and convergence behavior are monitored. Each experiment’s configuration and results are tracked with an experiment management tool. Automating repeatable training pipelines prevents manual errors and ensures reproducibility. A robust training loop lays the groundwork for reliable predictions.

4. Evaluation

Evaluation measures how well a model performs on unseen data. Metrics should align with real-world goals—accuracy, precision/recall, RMSE, AUC, or custom business KPIs. A hold-out test set that never touched training or tuning is reserved to avoid leakage. Performance is visualized with confusion matrices or ROC curves to spot strengths and weaknesses. Error cases are drilled into to understand model blind spots. Bias is checked by comparing results across different subgroups. Findings are documented clearly to guide stakeholders on strengths and limitations. Thorough validation builds confidence before production rollout.

5. Deployment

Deployment turns a validated model into a usable service or batch task. The model is containerized with Docker or packaged for serverless platforms. Inference logic is wrapped in a simple API or integrated into an existing data pipeline. Model artifacts and API definitions are version-controlled to track changes over time. Deployment pipelines are automated to eliminate manual hand-offs and reduce errors. Endpoints are load-tested to verify they meet latency and throughput requirements. Fallback logic or circuit breakers are included to handle unexpected failures gracefully. Smooth, repeatable deployment processes make going live painless and predictable.

6. Monitoring

Once in production, a model enters a dynamic environment that can drift over time. Input feature distributions are monitored to catch covariate drift early. Performance metrics are tracked against expected baselines or business KPIs continuously. Logs are instrumented for latency, error rates, and resource usage. Alerts are set to trigger when metrics deviate beyond safe thresholds. Diagnostic dashboards are used to spot anomalies at a glance. Regular health checks ensure the service remains stable under varying loads.

7. Maintenance

Maintenance keeps a model effective as data and requirements evolve. Over time, input distributions shift, feature relevance changes, and new business rules emerge. A proactive maintenance plan ensures the pipeline doesn’t silently degrade.

A retraining cadence is established based on data volume and observed drift. Every model artifact—training code, feature transformations, and weights—is versioned to allow rollback or comparison of historic performance.

Tests are automated to validate feature pipelines and inference logic after updates. New data sources or labels are incorporated when business needs shift, then their impact is validated through the evaluation suite. Obsolete features are rotated out and their effect on performance monitored to prevent bloat.

Wrapping Up

A robust machine learning pipeline weaves together data collection, feature engineering, model training, evaluation, deployment, monitoring, and maintenance. Each stage builds on the last to create a transparent, reproducible path from raw inputs to reliable predictions. Skipping any phase invites hidden bias, untraceable errors, and brittle systems.

When this end-to-end flow is followed; the right data is captured with clear ownership. That data is distilled into features that carry meaningful signals. Models are trained under controlled conditions and validated on truly unseen cases. Deployment is handled with repeatable pipelines, version control, and safety nets. Monitoring is continuous, adapting to real-world drift rather than hoping nothing breaks.

With this structure in place, Engineers can iterate on features or algorithms the team can focus on building next-generation capabilities. More importantly, Stakeholders can see clear metrics that connect model outputs to business impact.

Share ValueCurve

Subscribe now

Core Plotting and Customization

Keith Galli, a MIT graduate has published over 100 videos on Computer Science, Programming & Data Analysis. Keith created Python plotting crash course using matplotlib for beginners, few years ago. This course begins with the fundamentals of plotting before moving into customization.

• Setup: Import the libraries: matplotlib.pyplot as plt, numpy as np, and pandas as pd.

• Basic Line Graph: Create a simple line graph with plt.plot(x, y) and display it using plt.show().

• Labels and Titles: Add context with plt.title(), plt.xlabel(), and plt.ylabel().

• Tick Marks: Adjust axis tick values with plt.xticks() and plt.yticks() for readability.

• Legends: Use plt.legend() to label multiple datasets; it will auto-place the legend.

• Line Styling: Control appearance with parameters such as color, linewidth, marker, markersize, and linestyle.

• Shorthand Notation: Combine style settings quickly, e.g. ‘b.-’ for a blue line with dots.

• Font Customization: Pass a fontdict to titles or labels to change size, family, and weight.

• Figure Size and Resolution: Define figure dimensions with plt.figure(figsize=(w, h), dpi=300).

• Saving a Plot: Export graphics with plt.savefig(’plot.png’, dpi=300).

Variety of Plot Types

The course covers multiple visualization methods beyond line graphs:

• Line Graphs — e.g., gas price trends.

• Bar Charts — plt.bar(), with bar patterns via set_hatch().

• Histograms — plt.hist() to show data distributions, with adjustable bins.

• Pie Charts — plt.pie(), including labels, percentages (autopct), custom colors, and slice separation (explode).

• Box and Whiskers Plots — plt.boxplot() for comparing distributions, with options to style boxes.

Real-World Application with Pandas

Integration with Pandas is a key feature:

• Loading Data: Read CSVs with pd.read_csv().

• Example Datasets:

  • Gas Prices — country-level prices over time.

  • FIFA Player Stats — player attributes from the FIFA game.

• Data Selection: Extract subsets before plotting, such as gas[’USA’] or filtering with FIFA.loc[FIFA[’Club’] == ‘FC Barcelona’].

Teaching Approach & Takeaway

The course keeps a practical focus:

• Documentation First: Suggests to use the official Matplotlib docs for functions and parameters.

• Problem Solving: Search Google or Stack Overflow for specific issues, as is common in real development. This can now be augmented with AI tools.

This course covers the essentials of Matplotlib: how to create plots, customize them, work with multiple chart types, and apply them to real datasets through Pandas. It’s a practical introduction for anyone learning data visualization in Python. The datasets required to complete the matplotlib course can be downloaded from github . Additionally, you can also download cheatsheets for reference from the official site.

Share ValueCurve

Subscribe now

1. Huawei: Ascend 910C Chip and Roadmap

Huawei began shipping its Ascend 910C chip to major firms like Baidu and ByteDance for testing. Claimed to match Nvidia’s H100 in performance, this domestic GPU breakthrough supports self-reliant AI infrastructure, with initial deliveries enabling faster training of large models on Chinese hardware. Huawei also announced its 2025-2028 Ascend roadmap, incorporating self-developed HBM memory for 2 PFLOPS per chip to ensure supply-chain independence, with clusters matching NVIDIA’s aggregate power despite single-chip gaps.

2. Alibaba: Qwen3-Max Model Leadership

Building on this hardware foundation, Alibaba made headlines with its Qwen3-Max, a 1T+ parameter model that topped Hugging Face rankings. Backed by a ¥380B AI infrastructure investment, it surpasses Llama 3.1 in math and coding tasks, supports 29+ languages, and offers a massive 2M-token context window. The Qwen3-Max-Instruct variant targets coding, instruction-following, and agent applications, underpinning rapid enterprise adoption and closing the U.S.-China benchmark gap.

3. Tencent: Hunyuan Image 3.0 Multimodal Model

Meanwhile, Tencent contributed advancements on the multimodal front by releasing Hunyuan Image 3.0, an 80-billion-parameter Mixture-of-Experts model. Unlike standard diffusion models, it uses a unified autoregressive framework to tightly fuse text and image generation, excelling in photorealism and creative control while broadening accessibility for AI content creation across industries.

4. DeepSeek: V3.2-Exp Efficiency Upgrade

Efficiency and cost-effectiveness received attention as well, with DeepSeek unveiling its V3.2-Exp intermediate model. Building on prior versions, it introduces DeepSeek Sparse Attention to enhance long-context processing while halving API costs. Optimized for Chinese native chips and supporting CUDA cross-compatibility, it positions itself as a flexible, cost-competitive alternative in the LLM market.

5. Zhipu AI: GLM-4.6 and Claude Migration Plan

Zhipu AI also advanced with GLM-4.6, enhancing long-context reasoning and agent workflows while supporting massive input and output token windows. Available via API and open weights for local deployment, Zhipu’s release comes with a migration plan targeting Anthropic Claude users—offering a lower-cost, higher-usage alternative designed to attract a large user base.

6. ByteDance: Doubao 1.6-Vision Multimodal Launch

Finally, ByteDance’s cloud and AI unit Volcengine launched Doubao 1.6-Vision, a multimodal model introducing tool-calling for complex visual tasks. It delivers advanced visual reasoning and image operation capabilities like cropping and annotation, while slashing deployment costs nearly in half compared to its predecessor, enabling broader affordability and scalability for visual AI services.

Taken together, these announcements show a coordinated acceleration of China’s AI ecosystem—expanding from hardware foundations to frontier models and cost-efficient applications. The focus on domestic chip capability, long-context LLMs, multimodal systems, and accessible pricing signals more than just catch-up; it highlights a maturing ecosystem increasingly able to set competitive benchmarks on its own terms.

Share ValueCurve

Classes and Objects

• Object: Everything in Python is an object, and every object has a type that defines its data and behaviors.

• Class: A class acts as a blueprint for creating objects.

• Instance (Object): A specific creation based on that class blueprint.

Defining a Class

You define a class using the class keyword, followed by the class name. In Python 3, it’s common to write class Coordinate: instead of class Coordinate(object): because all classes inherit from object by default.

Attributes

• Data attributes (instance variables): Values stored per object, typically defined in __init__.

• Methods: Functions defined inside a class that operate on its instance data.

__init__ Method and self

• Constructor (__init__): Called automatically when a new instance is created; initializes the object’s attributes.

• self parameter: Refers to the instance on which a method is called. It is passed implicitly; self is the convention, not a Python keyword.

Data Attributes

Defined inside __init__ with self (e.g., self.x = xval). These persist for the object’s lifetime.

Dunder Methods and Operator Overloading

• Special Methods (Dunder Methods): Begin and end with double underscores, such as __init__, __str__, __add__.

• __str__: Defines a human-readable string (used in print() calls). Falls back to __repr__ if not defined.

• Operator Overloading: By defining methods like __add__, __eq__, and __mul__, you control how operators work on your custom objects.

Inheritance

• Concept: A child (subclass) can inherit attributes and methods from a parent (superclass), e.g., class Cat(Animal):.

• Overriding: A child can replace a parent’s method with its own definition.

• Extending: A child can add new attributes/methods.

• Method Resolution Order (MRO): Python looks for attributes in the current class, then parent(s), following a consistent MRO for multiple inheritance.

Class Variables

• Variables defined inside the class but outside methods are shared by all instances.

• Be careful: assigning to self.var in an instance will create/override an instance attribute, shadowing the class variable.

  Thanks for reading ValueCurve! This post is public so feel free to share it.

  Share

Python Code Example: Defining a Coordinate Class

This code block demonstrates how to define a class, create a constructor (__init__), define data attributes, and implement a procedural method (distance). It also shows the power of the __str__ method.

import math

class Coordinate:
    # Class Variables (shared by all instances)
    dimension = 2
    instance_count = 0

    def __init__(self, x_val, y_val):
        # Instance Variables
        self.x = x_val
        self.y = y_val
        Coordinate.instance_count += 1

    def distance(self, other_coord):
        “”“Calculates the Euclidean distance to another Coordinate.”“”
        x_diff = self.x - other_coord.x
        y_diff = self.y - other_coord.y
        return math.sqrt(x_diff**2 + y_diff**2)

    def __str__(self):
        “”“Defines a human-readable string representation.”“”
        return f”<{self.x}, {self.y}>”

    def __add__(self, other_coord):
        “”“Overloads ‘+’ operator to add coordinates (vector addition).”“”
        new_x = self.x + other_coord.x
        new_y = self.y + other_coord.y
        return Coordinate(new_x, new_y)

# --- Creating Objects ---
point1 = Coordinate(3, 4)
point2 = Coordinate(0, 0)
point3 = Coordinate(10, 5)

# --- Accessing Attributes and Methods ---
print(f”Point 1 X-attribute: {point1.x}”)
print(f”Distance between {point1} and {point2}: {point1.distance(point2):.2f}”)
print(f”Point 3 representation: {point3}”)

# --- Operator Overloading ---
point_sum = point1 + point3
print(f”Point Sum (point1 + point3): {point_sum}”)

# --- Class Variables ---
print(f”Total Coordinate objects created: {Coordinate.instance_count}”)
print(f”Dimension (class variable): {point1.dimension}”)

Summary and Best Practices

OOP provides the tools to manage large, dynamic projects by modeling complexity with clarity, structure, and consistency. Mastering these fundamentals is the key to advancing beyond basic scripting into professional software development.

The central mechanisms are:

• Instantiation: The __init__ method and the self parameter construct new objects and bind instance-specific data.

• Specialization: Inheritance allows creation of new classes that extend or override the behavior of their parents, promoting code reuse and clarifying relationships.

• Integration: Dunder methods like __str__ and __add__ let custom objects integrate naturally with Python’s built-in functions and operators, producing more readable and “Pythonic” code.

Best practices include:

• Designing classes around a clear, single responsibility to keep them maintainable and understandable.

• Using __str__ for human-readable output and __repr__ for accurate, developer-facing representations.

• Managing class variables carefully, recognizing when values should be shared at the class level versus kept per instance.

• Favoring composition (building complex objects from simpler ones) when it improves clarity, rather than relying heavily on inheritance hierarchies.

• Applying operator overloading selectively, only when it genuinely improves code clarity and aligns with intuitive object behavior.

OOP is powerful in Python but should be used thoughtfully, alongside procedural and functional styles, to build flexible, scalable, and maintainable systems.

Subscribe now

Share ValueCurve

Key Characteristics and Operations

Mutability: The most significant feature of a list is its mutability. Unlike immutable objects such as strings and tuples, lists can be changed in memory after they are created. This means you can alter a list object itself without creating a new copy. This is a key difference from tuples, where reassigning a variable to a new tuple creates a new object in memory, leaving the original unchanged. Because of their mutable nature, lists are highly efficient for managing dynamic data, like a list of employees or students, as changes don’t require creating entirely new copies of large data structures.

Modifying Elements: You can change an element at a specific index using assignment syntax, such as L[index] = 5.

Adding Elements:

• append(): Adds a single item to the end of a list, mutating the original list. Returns None.

• extend(): Adds all elements from another list to the end of the original.

Removing Elements:

• remove(element): Removes the first occurrence of a specified value.

• del L[index]: Deletes the element at a specific index.

• pop(): Removes and returns the last element.

• clear(): Removes all elements, keeping the same list object in memory.

Sorting and Reversing:

• L.sort(): Sorts the list in place, returning None.

• L.reverse(): Reverses the list in place.

• sorted(L): Returns a new, sorted copy without altering the original.

Iteration:

• Iterate directly with for e in L: (more Pythonic).

• Use for i in range(len(L)): if you need to mutate elements by index.

• Mutating a list while iterating directly can cause skipped elements; safer: iterate over a copy.

Copying (Cloning):

• Shallow copy: L_copy = L[:] (top-level only).

• For independent nested structures, use copy.deepcopy(L).

List Comprehension:

• Pattern: [expression for item in sequence if condition].

• A concise way to replace explicit loop + append structures.

Summary and Best Practices

In summary, the Python list is a versatile and indispensable collection. To use it effectively and avoid common pitfalls, keep these core principles in mind:

• Embrace Mutability: Use in-place methods like append(), extend(), and sort() for efficient data alteration, but be mindful that these methods always mutate the original list.

• Beware of Aliasing: Use shallow copy techniques such as L[:], list(L), or L.copy() when you need an independent copy of a list’s top-level structure. This prevents unintentional modifications to the original list. If the list contains nested mutable structures (e.g., lists of lists), use copy.deepcopy(L) when you need a fully independent copy of both top-level and nested objects.

• Use List Comprehensions: Adopt the [expression for item in sequence if condition]pattern to write cleaner, faster, and more readable code when generating new lists.

• Prefer Safe Mutations During Iteration: Mutating a list while iterating can cause skipped elements. Iterate by index or over a copy for safety.

• Understand Sorting Options: Use list.sort() when you want to sort in place for performance. Use sorted(list) when you need a new sorted copy without affecting the original.

  Share ValueCurve

Subscribe now

Konstantine Buhler from Sequoia draws a compelling parallel between milestones of the industrial era—the steam engine, the first factory system, and the assembly line—and key developments in the AI era, such as the introduction of the first GPU (the GeForce 256 in 1999) and the establishment of the first “AI factory” in 2016. Just as it took 144 years to perfect the factory assembly line, AI is now entering a crucial phase of specialization.

For a complex system like AI to mature, it must integrate general-purpose components (like foundational AI models) with highly specialized subsystems and labor. Sequoia views today’s startups as the primary drivers of this specialization, building targeted applications atop general AI technologies.

How the AI Future will unfold

Sequoia expects AI to catalyze a major economic shift by automating and expanding the market for knowledge work and services. They identify the $10 trillion US services market as the primary opportunity. Comparing it with the evolution of software-as-a-service (SaaS)—which expanded the on-premise software market—AI is anticipated not only to grow the market share but also to enlarge the services industry itself. This expansion could give rise to large, standalone public companies centered on AI, akin to the industrial giants of past eras.

Investment Trends Sequoia is watching

1. Leverage Over Uncertainty: Work is shifting from tasks with low leverage and high certainty to tasks where AI delivers massive leverage (100%+), albeit with less predictable outcomes. For example, a salesperson might deploy hundreds of AI agents to monitor accounts and intervene only as needed.

2. Real-World Measurement: The benchmark for AI performance has moved beyond academic datasets like ImageNet. Sequoia points to real-world validation, such as AI hackers competing live on platforms like HackerOne, as a more meaningful measure.

3. Reinforcement Learning: Previously limited to research labs, reinforcement learning is now employed by startups to train open-source models, especially in coding and software development.

4. AI in the Physical World: AI is expanding beyond software, powering robotics, manufacturing processes, and quality assurance systems.

5. Compute as the New Production Function: The emerging key metric is “flops per knowledge worker.” Sequoia’s portfolio companies forecast a 10x to 10,000x increase in compute consumption as workers begin leveraging hundreds or thousands of AI agents.

Investment Themes for the next 12–18 Months

1. Persistent Memory: A significant unsolved challenge in AI is long-term memory—both in retaining conversational context and preserving an agent’s identity. Current approaches, including vector databases and extended context windows, remain insufficient & have to be addressed.

2. Seamless Communication Protocols: Just as TCP/IP enabled the internet, new protocols are needed for AI agents to communicate and collaborate effectively. This advancement would allow agents to perform complex tasks, such as researching, comparing prices, and completing purchases autonomously.

3. AI Voice: Advances in fidelity and reduced latency now make AI voice suitable for real-time conversations. Applications include consumer-facing virtual companions and enterprise solutions like logistics coordination and trading desks.

4. AI Security: There is a substantial opportunity to embed security at every layer of the AI stack—from model development to end-user protection. Sequoia envisions a future with hundreds of AI security agents safeguarding each human and AI agent.

5. Open Source: Despite its precarious position, Sequoia considers open-source AI essential for a free and equitable future. They aim to support models that remain accessible and competitive, counterbalancing proprietary dominance.

Sequoia’s view is that AI is entering a phase of deep specialization, with startups leading the development of the next wave of infrastructure and applications. This moment represents an economically transformative era with long-term opportunities across services, compute, security, and open-source ecosystems.

Share ValueCurve

Subscribe now

The need for dictionaries arises from the limitations of using lists for complex data storage. Let’s consider this with a student grades example:

• Using parallel lists (one for names, one for grades) is cumbersome. To find a student’s grade, you first have to find their index in the names list and then use that same index to access the grades list. This process becomes increasingly messy as more data (like quiz or problem set scores) is added, requiring more parallel lists that must be kept in sync.

• Using a “master list” of nested lists also proves to be complex and hard to read, requiring nested loops to search for specific information.

Dictionaries solve this by providing a direct mapping from a key (e.g., a student’s name) to a value (e.g., their grade or a more complex data structure containing all their information). An entry in a dictionary is a key-value pair.

Creating, Accessing, and Modifying Dictionaries

Creation

• Dictionaries are created using curly braces {}. An empty dictionary is created with d = {}.

• Entries are defined as key: value, and multiple entries are separated by commas. For example, a dictionary mapping names to grades could be created like this: grades = {’Ana’: ‘B’, ‘Matt’: ‘A’, ‘John’: ‘B’}.

Accessing and Mutability

• Dictionaries are mutable objects, meaning they can be changed after they are created.

• To look up a value, you use the key as an index in square brackets, similar to list indexing (e.g., grades[’John’] would return ‘B’).

• If you try to access a key that does not exist in the dictionary, Python will raise a KeyError exception.

• You cannot look up a key by its value directly, as values can be duplicated. You would need to write a loop to perform this reverse lookup.

Operations

• Adding or Modifying Entries: The same syntax is used for both adding a new entry and changing an existing one. If the key does not exist, a new entry is created. If it does exist, its value is overwritten. For example, grades[’Grace’] = ‘A’ adds Grace, and grades[’Grace’] = ‘C’ later changes her grade.

• Deleting Entries: The del keyword is used to remove an entry, such as del grades[’Ana’]. This mutates the original dictionary.

• Checking for a Key: The in operator checks for the existence of a key in a dictionary (e.g., ‘John’ in gradesreturns True). It is important to note that the in operator only checks keys, not values.

• Copying: Since dictionaries are mutable, simple assignment (d2 = d1) creates an alias (another name for the same object). To create a shallow copy, you must use the .copy() method (d2 = d1.copy()).

  ---

Iteration

While recent versions of Python guarantee insertion order for dictionaries, it is best practice to write code assuming they are unordered sequences for robustness and compatibility with older versions. There are three main ways to iterate over a dictionary:

• .keys(): Returns an iterable sequence of all the keys in the dictionary.

• .values(): Returns an iterable sequence of all the values.

• .items(): This is often the most effective method, as it returns an iterable of key-value pairs as tuples. This allows you to access both the key and value simultaneously in a loop, for example: for k, v in my_dict.items():.

Restrictions on Keys and Values

• Dictionary values can be of any type, including other mutable objects like lists or other dictionaries. Values can also be duplicated.

• Dictionary keys have two main restrictions:

  1. They must be unique. You cannot have two entries with the same key.

  2. They must be immutable (technically, “hashable”). This means keys can be types like integers, floats, strings, tuples, and booleans, but cannot be lists or other dictionaries.

Performance and Implementation: Hashing

Following is a detailed explanation of why dictionaries offer superior performance for lookups compared to lists and why keys must be immutable :

• Average-Case constant time lookup: Dictionaries are implemented using a hash table. A hash function is run on a key, which converts it into an integer. This integer is then used as a direct index to find the location of the corresponding value in the hash table (which is structured like a list in memory). Because this lookup is based on a direct calculation rather than a sequential search, accessing an item in a dictionary takes constant time (Θ(1)) on average. This provides a massive performance advantage over searching a list, which takes linear time (Θ(n)).

• Collisions and worst-case performance: A “collision” occurs when two different keys produce the same hash value, mapping them to the same location or “bucket” in the hash table. These collisions are handled by storing the colliding entries together, often in a list-like structure within that bucket. A good hash function distributes keys uniformly, minimizing collisions. However, in the worst-case scenario, if all keys collide into the same bucket, looking up an item degrades to a linear search through that bucket, resulting in linear time (theta(n)) performance.

• Why keys must be immutable: The hashing mechanism requires that a key always produces the same hash value. If a key were a mutable object like a list, its contents could change after it was stored in the dictionary. This change would alter its hash value, making it impossible for the dictionary to find the original location where the value was stored.

Use Cases

1. Storing Structured Data: Dictionaries are excellent for representing complex, structured data. The student information example was revisited to show how a nested dictionary (grades[’Ana’][’ps’]) provides a much cleaner and more efficient way to store and retrieve data compared to nested lists.

2. Counting Frequencies: A common pattern is to create a “frequency dictionary” to count occurrences of items. An example shows how to build a dictionary that maps each word in a song’s lyrics to the number of times it appears.

3. Memoization: Dictionaries are crucial for an optimization technique called memoization, which is used to speed up expensive computations, particularly in recursive functions. The Fibonacci example demonstrates this by storing previously calculated fib(n) values in a dictionary. Before computing a value, the function checks if the result is already in the dictionary, avoiding redundant calculations. This improved the efficiency of the Fibonacci function from exponential time to a much faster one, reducing millions of function calls to just a few dozen.

Summary and Best Practices

Python dictionaries are the core tool for managing unordered, dynamic data that requires fast, key-based lookup. They fundamentally map unique, hashable keys to corresponding values.

The dictionary’s defining characteristic is its reliance on hash tables, which allows for item retrieval in average-case constant time (Θ(1)), offering a massive performance advantage over sequential list searching.

Share ValueCurve

Subscribe now

Learning to program is a skill that requires consistent practice. It is not just about understanding concepts but also about developing problem-solving ability—the process of translating a problem described in English into a computational solution. Following concepts should serve as a memory aid for the beginners in Python.

Objects, Types, and Variables

• Objects and Types: In Python, everything is an object, from numbers to functions. Each object has a type that defines what operations can be performed on it. You can use type()to check an object’s type.

• Scalar Data Types:

  • int: Whole numbers like 5, 0, -100.

  • float: Real numbers with a decimal point, such as 3.27 or 2.0. Because of binary representation, floating-point operations may introduce tiny rounding errors.

  • bool: Truth values—True or False (case-sensitive).

  • NoneType: Has only one value, None, used to represent the absence of a value.

• Variables and Expressions:

  • Variables are names bound to objects. The assignment operator = binds the name on the left to the evaluated value on the right.

  • Example: area = pi * radius**2 first evaluates the expression and then assigns it to area.

  • Variables can be rebound to new values. For instance, x = x + 1 calculates x + 1 and rebinds x to this result.

• Operators:

  • Arithmetic: +, -, *, **, / (float division), // (integer division), % (modulo).

  • Comparison: ==, !=, >, < evaluate to True or False.

  • Logical: and, or, not.

Input, Output, and Control Flow

• Input and Output:

  • print() displays output.

  • input() reads user input as a string. Casting is often required, e.g., age = int(input(”Enter age: “)).

  • f-strings provide formatted string interpolation: print(f”Area = {area}”).

• Control Flow:

  • Branching: if, elif, else allow decision-making. Indentation defines code blocks.

  • Loops:

    • while: Repeats while a condition is True. Be careful to avoid infinite loops.

    • for: Iterates over sequences. Example: for i in range(5):.

Functions

• Defining and Calling: Declared with def and executed when called.

• Parameters and Return Values: Accept inputs and return results with return. Functions without a return default to None.

• First-Class Objects: Functions can be assigned to variables, passed as arguments, or returned from other functions.

• Scope: Each function has its own local scope.

  • Local variables are destroyed after the function finishes.

  • Functions can read variables from outer scopes but cannot modify them unless declared with global or nonlocal.

• Loop Control:

  • break: Exit the loop early.

  • continue: Skip to the next iteration.

  • pass: Placeholder that does nothing (useful for code stubs).

Common Data Structures

• Strings (str): Immutable, ordered sequence of characters.

  • Created with quotes: “hello”.

  • Operations: concatenation (+), repetition (*), length (len()), indexing, slicing (s[1:3]).

• Tuples (tuple): Immutable, ordered collection of objects.

  • Example: (2, ‘MIT’, 3).

  • Support indexing, slicing, and unpacking (x, y = y, x).

  • Often used to return multiple values from a function.

• Lists (list): Mutable, ordered collection.

  • Example: [2, ‘a’, 4].

  • Elements can be modified: L[0] = 5.

  • Common methods:

    • L.append(item) – add item at end

    • L.extend(other) – append all items from another list

    • L.sort() – sort list in place

    • L.reverse() – reverse in place

    • L.remove(item) – remove first matching element

  • Assigning newList = oldList creates an alias, not a copy. For a copy: newList = oldList[:].

• Dictionaries (dict): Mutable collection of key-value pairs.

  • Example: grades = {’Ana’: ‘B’, ‘John’: ‘A’}.

  • Keys must be immutable types (str, int, tuple).

  • Access: grades[’Ana’].

  • Add or modify: grades[’Grace’] = ‘A’.

  • Iteration: .keys(), .values(), .items().

  • Safe access: grades.get(’Ana’, ‘Not Found’).

Exception and Error Handling

Python provides robust mechanisms to handle errors using try, except, and finally blocks. While not essential at the very beginning, learning exceptions is important once you start writing programs that deal with unpredictable input, files, or external data.

Summary & Best Practices

Python is an imperative, object-oriented language in which every entity is treated as an object. Its core components include variables, operators, control flow mechanisms, data structures, and functions.Mutability plays a key role: lists and dictionaries are mutable, while strings and tuples are immutable. Functions support abstraction, modularity, and reuse, with built-in features like default and keyword arguments.

Clean Python code is easier to read and maintain when variable names clearly reflect their purpose. For loops are typically more readable than while loops, unless the loop depends on a specific condition and f-strings simplify string formatting by embedding expressions directly. Mutable objects like lists and dictionaries can lead to unintended side effects when shared across functions or scopes, so they require careful handling.

Deeply nested logic can obscure intent; breaking it into smaller functions improves clarity and reuse. User input should be converted and validated to prevent errors. Comments are useful when necessary, but well-structured code and meaningful names often make them redundant. By combining a solid grasp of the language’s core constructs with disciplined coding practices, developers can build systems that are both elegant and resilient.

Share ValueCurve

[1] Anthropic has released Claude Sonnet 4.5, its most capable model to date, designed for complex agentic workflows and high-stakes coding tasks. The model supports a 200K token context window by default, with access to a 1 million token context in beta, enabling long-horizon tasks and large document processing.

As per Anthropic, Sonnet 4.5 demonstrates stronger performance in reasoning, code generation, and multimodal understanding. It offers a balance of intelligence and speed suitable for enterprise workloads and real-time AI experiences and the alignment has been significantly improved. The model now shows reduced tendencies toward sycophancy, deception, and power-seeking, and is more robust against prompt injection attacks. Sonnet 4.5 is available via the Claude API, Amazon Bedrock, and Google Cloud’s Vertex AI, with pricing unchanged from Sonnet 4.

[2] OpenAI has introduced parental controls for ChatGPT to help families manage teen usage in a safer, age-appropriate way. Parents can link their accounts with their teen’s, customize settings, and apply safeguards such as reduced exposure to graphic content, sexual or violent roleplay, and viral challenges. Additional controls include quiet hours, disabling voice mode, memory, image generation, and opting out of model training. A notification system alerts parents if ChatGPT detects signs of potential self-harm. As per Open AI, these features were developed in consultation with experts and advocacy groups, and are part of OpenAI’s broader effort to build toward an age prediction system that automatically applies teen-appropriate settings.

Subscribe now

[3] OpenAI has launched “Buy it in ChatGPT,” starting with Instant Checkout, a feature that lets users purchase products directly within ChatGPT. Initially available for U.S. Etsy sellers and expanding soon to over a million Shopify merchants like Glossier, SKIMS, and Vuori , it supports single-item purchases with multi-item carts coming later. The feature is powered by the Agentic Commerce Protocol (ACP), an open standard co-developed with Stripe. ACP enables secure, real-time communication between AI agents (like ChatGPT), buyers, and merchants, allowing ChatGPT to act as a digital shopper that facilitates transactions without leaving the chat.

[4] Gemini Robotics-ER 1.5 is Google DeepMind’s latest embodied reasoning model, designed to serve as a high-level cognitive engine for physical agents. It integrates spatial and temporal reasoning, task planning, and progress estimation, enabling robots to interpret scenes, plan multi-step actions, and execute tasks using external tools like Google Search or custom APIs. The model introduces a tunable “thinking budget” to balance latency and accuracy, supports semantically grounded 2D point generation, and demonstrates improved safety by recognizing physical constraints and refusing unsafe plans. Gemini Robotics 1.5 refers to the broader system that combines ER 1.5 with vision-language-action (VLA) models and cross-embodiment learning to support end-to-end robotic control. Gemini Robotics-ER 1.5 is available in preview via the Gemini API in Google AI Studio, while Gemini Robotics 1.5 is currently accessible only to select partners.

[5] Looker Studio is Google Cloud’s self-service business intelligence platform for creating customizable dashboards and reports. It supports over 800 data connectors and includes features like drag-and-drop editing, prebuilt templates, report embedding, and an API for asset management. Recent updates include the launch of Looker Studio Pro, which adds enterprise capabilities such as team workspaces, Google Cloud project linking, and admin support for governance and access control. Pricing for Looker Studio Pro is $9 per user per project per month, while the standard version remains free for creators and viewers.

[6] Tilde (an EU / Baltic / Nordic AI startup) released TildeOpen LLM, a 30-billion-parameter open model optimized for European languages and trained on the EuroHPC LUMI supercomputer. TildeOpen was trained on the LUMI supercomputer using AMD Instinct™ MI250X accelerators and supports all 24 official EU languages plus others, and is released under a permissive license (CC-BY-4.0). This bolsters Europe’s multilingual AI infrastructure, improving coverage in lesser-served languages.

Subscribe now

Tuples as Data Structures

Tuples are a fundamental Python data structure—a way of organizing data with a focus on immutability and reliability. Unlike lists, tuples guarantee their contents won’t change, which makes them suitable in contexts such as dictionary keys, fixed records, or multiple return values

Share

Tuples: Immutable Sequences

A tuple is an ordered collection of values, typically enclosed in parentheses `()`. Unlike lists, tuples cannot be modified once defined.

coordinates = (10, 20)
# Trying to change an element raises an error:
# coordinates[0] = 99   # TypeError: ‘tuple’ object does not support item assignment

This immutability makes tuples ideal for storing fixed collections of related data where consistency matters.

Packing: Collecting Values into a Tuple

Packing is a feature where multiple values are grouped together into a tuple automatically.

point = 5, 7
print(point)   # Output: (5, 7)

This allows for easy grouping of values without explicit tuple syntax.

Unpacking: Assigning Tuple Elements

Unpacking takes a tuple and assigns its elements to individual variables in one step.

x, y = point
print(x, y)   # Output: 5 7

Unpacking is useful when functions return multiple values or when processing tuple-based data.

Subscribe now

`zip`: Combining Iterables

The `zip()` function pairs elements from multiple iterables into tuples, providing a combined view of the data.

names = [”Alice”, “Bob”, “Charlie”]
scores = [85, 90, 95]

zipped = zip(names, scores)
print(list(zipped))  
# Output: [(’Alice’, 85), (’Bob’, 90), (’Charlie’, 95)]

This is helpful for parallel iteration over multiple sequences.

`enumerate`: Indexed Iteration

When you want both the index and the value from a sequence, `enumerate()` produces tuples of `(index, value)` pairs.

fruits = [”apple”, “banana”, “cherry”]

for index, fruit in enumerate(fruits, start=1):
    print(index, fruit)

# Output:
# 1 apple
# 2 banana
# 3 cherry

Sorting with a Key

Tuples can represent structured data, such as `(name, score)` pairs. When sorting collections of tuples, a sort key can determine the criterion.

students = [(”Alice”, 85), (”Bob”, 90), (”Charlie”, 78)]

sorted_students = sorted(students, key=lambda student: student[1])
print(sorted_students)
# Output: [(’Charlie’, 78), (’Alice’, 85), (’Bob’, 90)]

This sorts the list based on the score (the second tuple element).

Combined with Python’s powerful features like packing, unpacking, `zip`, and `enumerate`, tuples provide a simple but robust way to handle fixed collections of data efficiently and safely.

Stay Connected

Subscribe now

The Challenge

The task is to write a program that, given two lowercase phrases (with spaces and punctuation removed), determines whether they are anagrams. For instance, listen and silent are anagrams, while hello and world are not. Though often used as a beginner coding exercise, this challenge has real-world relevance in systems that require fast, reliable string comparison—such as search engines, data deduplication, and secure identifier matching.

Counting Characters

Two strings are anagrams if and only if they contain the same characters with the same frequency. For example, both listen and silent contain one l, one i, one s, one t, one e, and one n. If the character counts match for all letters, the strings are anagrams. This principle forms the basis of the program’s logic: rather than comparing order, it compares composition.

Algorithm in Three Steps

1. Length Check: Return false immediately if the lengths differ, as they cannot be anagrams.

2. Frequency Counting: Iterate over both strings, counting characters using dictionaries.

3. Comparison: Compare the two dictionaries; if they match, print "true", else "false".

This approach ensures efficiency and clarity, especially when scaled to multiple test cases.

Python Code

def solve_anagrams():
    try:
        print("Anagram Verifier")
        print("----------------")
        print("Enter a number between 1 and 20 for anagram pairs to check")
        t = int(input())  # Number of anagram pairs to check 
        if not (1 <= t <= 20):
            return

        for _ in range(t):
            line = input().split()
            s1, s2 = line[0], line[1]

            # Step 1: Length check
            if len(s1) != len(s2):
                print("false")
                continue

            # Step 2: Build character frequency dictionaries
            count1, count2 = {}, {}
            for char in s1:
                count1[char] = count1.get(char, 0) + 1
            for char in s2:
                count2[char] = count2.get(char, 0) + 1

            # Step 3: Compare dictionaries
            print("true" if count1 == count2 else "false")
    except (IOError, ValueError):
        return

solve_anagrams()

Practical Applications

This kind of string comparison has a wide range of uses. In security, anagram logic can help obscure data while preserving its verifiability. In games and puzzles, it’s a core mechanic for word challenges. In natural language processing, it helps in analyzing lexical variation and stylistic patterns. And in systems design, scrambling identifiers using anagram principles can enhance privacy without losing uniqueness.

While sorting both strings is also a valid method for checking anagrams, its performance degrades as input size grows. Sorting requires approximately n × log n steps (O(n log n) for those familiar with the Big O notation ), where n is the length of the string. Doubling the string length results in more than double the processing time. In contrast, counting character frequencies and comparing them takes only O(n) time, since it involves a single pass through each string. This linear-time approach is significantly faster and more scalable, making it the preferred method for high-performance applications.

To make the program more robust, it could be extended to handle cases such as Unicode or accented characters, which are common in multilingual datasets. From a performance standpoint, counting characters is generally more efficient than sorting, especially when dealing with long strings or large volumes of comparisons. For a cleaner implementation, Python’s collections.counter class offers a concise and powerful way to perform frequency-based comparisons with minimal code.

Share ValueCurve

The Problem with Narratives

The book’s premise is : most established narratives about India are fiction masquerading as fact. Despite having one of the world’s most impressive statistical architectures—Census enumerators, National Statistical Office surveyors, the National Crime Records Bureau—Indians continue to build inaccurate narratives about how the country works. They assume vote-banks where none exist, imagine urban explosions with little reason, and decide they are middle class with little basis. These narratives, repeated enough times, become political fodder.

Numbers, she argues, can capture the nuance and humanity that pre-packaged narratives flatten out.

Belief in God, Marriage & Love

It’s claimed 97% of Indians believe in God, with Buddhists as the notable exception. Buddhist traditions do not promote belief in a creator god, focusing instead on ethical conduct and discipline to attain nirvana—the cessation of suffering. While rituals exist in many Buddhist schools, they are not central to liberation. Religion remains important to Indians: 60% say they pray daily. Education level doesn’t affect religious practice. Indians are largely indifferent to, and tolerant of, the religious practices of others.

The book examines patterns in vegetarian vs. non-vegetarian food consumption, and how food habits intersect with upward mobility. It shows that modern India remains largely conservative—more orthodox in religious practice. It explores dietary choices, alcohol consumption, and the types of alcohol preferred. Foods like poultry, eggs, fish, fruits, and legumes—often more expensive—reveal distinct consumption patterns across income groups.

Indians overwhelmingly marry within their caste. The average age gap between husband and wife is around five years. The poorest 40% of women tend to marry before 18. Richer, better-educated women marry later. Over 90% of marriages in India are arranged by families. Love marriages are more common among richer, better-educated people, and among Christians and Muslims.

Work, Money & Leisure

The book explores how Indians earn, how much they earn, and how they spend on leisure. Enjoyment varies by caste, class, gender, and geography. Most women still aren’t paid for household labor. Nearly 40% of Scheduled Castes work as wage laborers—mostly in casual roles.

Riding motorbikes and singing chants are favourite leisure activities. Rich and upper-caste groups spend more time on religious practice and have more access to media. Inter-religious marriages are seen as acts of rebellion.

Fewer than 20% of women have their names on house papers; half have a bank account in their name; only 10% can make primary purchase decisions. For women working all day, talking to someone is the most meaningful leisure—because no one listens to them.

Crime Statistics: A Cautionary Tale

One of the book’s most striking revelations concerns crime data. Rukmini’s investigation of 600 rape cases in Delhi’s district courts revealed that a significant portion were not sexual assaults as commonly understood, but cases of consenting couples whose families had filed complaints to thwart inter-caste or inter-religious relationships. The FIRs followed informal scripts—”moving cars” that abducted young women, “sedative-laced cold drinks” that rendered victims unconscious. None of it stood up in court. The cold drink bottle was the smoking gun that never appeared.

This isn’t unique to sexual crime. Her investigation of Mumbai’s mephedrone cases found that police knowingly used wrong sections of law to arrest pedlars. The result: 100% acquittal rate, with nearly 150 young people spending over a year in jail on charges the police knew were legally unsound.

The takeaway: India’s crime statistics begin from a point of significant under-reporting, and states with higher reported crime might actually be doing a better job of ensuring full reporting rather than being the most unsafe.

Human Rights, Media & Voting Preferences

There are inferences that media exposure benefits the ruling BJP more than Congress, which performs better among those with low media access. Half of Indians vote based on caste. Many are open to strong leaders and tolerate media bias, discrimination, and segregation based on caste and religion.

Human rights and freedom of expression aren’t widely prioritized. From fans to cult followers, the idea of a fair judiciary or honest elections ranks low. Yet 70% still trust their state government. Media continues to reinforce bias. Among urban graduates and postgraduates, 66.6% lean toward ethnic nationalism and majoritarian views. Ghettoisation of minorities—especially Muslims—is rampant, from physical segregation to anti-Muslim rhetoric in mass media.

Voter turnout is around two-thirds, with more women voting than ever. The surge in female voters reflects growing participation. Ironically, the poor vote because it’s their right, while the non-poor vote for material benefits or out of a sense of duty.

Indians are on the move—especially from states like UP, Bihar, and Bengal. Migrants often can’t vote. In 2015, Janagraha, a Bangalore-based trust working in the areas of urban infrastructure and citizenship, reported that 11% of voter addresses couldn’t be found, 21% had moved, and nearly 50% couldn’t find their names on Delhi’s voter list. Pollsters and journalists often frame voting as driven by caste, religion, leaders, or development—but the reality is more nuanced. After the CAA protests, Muslims voted for AAP (not Congress), even though Congress supported the agitation—because AAP was more likely to defeat BJP. Notably, BJP has no elected Muslim representative.

The Data Imperative

The book’s conclusion carries urgency. Indian official statistics are not lying to us, Rukmini argues, but they are being silenced. A combination of neglect, discredit, and dismissal makes deficiencies seem too fatal to fix. When inconvenient data is suppressed, the narrative shipped to op-ed pages is that official statistics miss too much, so we’re better off without them.

This is dangerous. For 75 years, this data has shaped policy and driven change—debates about liberalisation, poverty lines, welfare states, affirmative action, rising Islamophobia. Access to this data has empowered ordinary citizens to engage with and agitate against the State. Without it, we hollow out democracy.

Final Thoughts

First published in 2021, the datasets referred in some cases may appear dated requiring an update, however the book is an interesting read for those who are curious about modern India—and essential for anyone who wants to engage critically with the narratives we’re fed.

Share ValueCurve

1. Installing Visual Studio Code

Visual Studio Code (VSCode) is a powerful, free code editor that's essential for development.

Download and Install:

1. Visit code.visualstudio.com

2. Click "Download for Mac"

3. Open the downloaded .zip file

4. Drag the Visual Studio Code app to your Applications folder

5. Launch VSCode from Applications or Spotlight

First Launch Setup:

• VSCode may ask for permissions to access folders - click "Allow"

• Install the Python extension by Microsoft from the Extensions panel (Cmd+Shift+X)

  

Subscribe now

2. Installing and Configuring Python

Python is the most popular language for AI development, offering extensive libraries and frameworks.

Install Python using Homebrew:

1. Open Terminal (press Cmd+Space, type "Terminal", press Enter)

2. First, install Homebrew (if not already installed):

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

3. Install Python:

brew install python

4. Verify installation:

python3 --version
pip3 --version

Configure Python Environment:

1. In Terminal, create a virtual environment for your projects:

python3 -m venv ~/ai-projects

2. Activate the environment:

source ~/ai-projects/bin/activate

3. Install essential packages:

pip install jupyter pandas numpy matplotlib requests

VSCode Python Setup:

1. Open VSCode and press Cmd+Shift+P

2. Type "Python: Select Interpreter"

3. Choose the interpreter from your virtual environment (~/ai-projects/bin/python)

Configure Shell Profile: In Terminal, add the virtual environment path to your shell profile for easy activation:

echo 'alias activate-ai="source ~/ai-projects/bin/activate"' >> ~/.zshrc
source ~/.zshrc

Now you can simply type activate-ai to activate your environment.

3. Installing and Configuring Claude CLI

Claude CLI enables direct interaction with Claude AI from your terminal, streamlining AI-assisted development.

Installation:

1. In Terminal, install Node.js (required for Claude CLI):

brew install node

2. Install Claude CLI globally:

npm install -g @anthropic-ai/claude-cli

Configuration:

1. Get your API key from console.anthropic.com

2. In Terminal, configure Claude CLI:

claude configure

3. Enter your API key when prompted

4. Test the installation:

claude --version

Basic Usage:

• Start a conversation: claude chat

• Get help with code: claude "explain this Python function" < myfile.py

• Generate code: claude "create a Python script to read CSV files"

  Share

4. Configuring VSCode with GitHub

GitHub integration enables version control and collaboration, essential skills for any developer.

Install Git: In Terminal, run:

brew install git

Configure Git: Still in Terminal:

git config --global user.name "Your Name"
git config --global user.email "your.email@example.com"

GitHub Authentication:

1. In Terminal, install GitHub CLI:

brew install gh

2. Authenticate with GitHub:

gh auth login

3. Follow the prompts to authenticate via browser

VSCode GitHub Setup:

1. Install the "GitHub Pull Requests and Issues" extension

2. Sign in to GitHub when prompted in VSCode

3. Open the Source Control panel (Cmd+Shift+G)

Creating Your First Repository:

1. Create a new folder for your project

2. Open it in VSCode: File > Open Folder

3. Initialize Git using VSCode's integrated terminal (`Ctrl+``):

git init

4. Create a proper .gitignore file in the VSCode terminal:

# Create .gitignore with common exclusionscat > .gitignore << EOF# Virtual environmentsvenv/env/.env# Python__pycache__/*.pyc*.pyo.Python# IDEs.vscode/settings.json.DS_Store# API keys and secrets.env.localconfig/secrets.json# Jupyter Notebooks.ipynb_checkpoints/EOF

5. Create a README file and make your first commit

6. Push to GitHub using the Source Control panel

Environment Variables Setup: For secure API key management, create a .env file in the VSCode terminal (already in .gitignore):

# Create .env file for secrets
echo "ANTHROPIC_API_KEY=your_api_key_here" > .env

Next Steps

This environment is optimized for AI development. The combination of VSCode's editing capabilities, Python's AI libraries, Claude CLI's assistance, and GitHub's collaboration tools provides everything needed to begin your AI fluency journey.

Remember to activate your Python virtual environment (activate-ai or source ~/ai-projects/bin/activate) each time you start a new terminal session for development work. Always keep sensitive information like API keys in .env files that are excluded from version control.

Leave a comment