Substrate

Infrastructure for Human Knowledge & Progress

What Is It • Data • Quick Start • Docs • Contribute • Roadmap

→ Watch Introduction Video • → Read Blog Post

Think of substrate as the base layer—the common ground where we can all work together to understand problems and drive solutions forward. Instead of fragmented discussions and reinventing the wheel, Substrate gives us one place to:

• Document problems → What's actually broken, with evidence
• Track solutions → What works, what doesn't, with results
• Connect progress → Link people, organizations, and projects actually moving things forward
• Build on each other → Arguments and ideas that build on shared evidence
• Measure outcomes → Did it work? What actually changed?

The Purpose: Accelerate Human Progress

We can't solve problems we don't understand. We can't build on solutions we can't find. Substrate provides:

• 🎯 Shared understanding → One place to understand what's wrong and what works
• 🚀 Faster progress → Build on existing knowledge instead of starting over
• 📊 Evidence-based action → Ground decisions in authoritative data, not opinions
• 🔗 Connected knowledge → See how problems, solutions, people, and data interconnect
• 🌍 Collective intelligence → Human insight + AI analysis working together toward progress

An open-source framework connecting 17+ knowledge components:

graph TB
    subgraph "🌍 The Substrate"
        A[🧩 Problems]
        B[💡 Solutions]
        C[📊 Data Sources]
        D[🗣️ Arguments]
        E[📋 Claims]
        F[👥 People]
        G[🏢 Organizations]
        H[🚀 Projects]
        I[📈 Plans]
        J[🎯 Values]
        K[💭 Ideas]
    end

    L[👤 Human Contributors] --> A
    L --> B
    L --> D
    M[🤖 AI Analysis] --> A
    M --> B
    M --> D

    A -.connects to.-> B
    B -.connects to.-> C
    D -.connects to.-> C
    E -.connects to.-> C
    F -.connects to.-> G
    G -.connects to.-> H
    H -.connects to.-> B

    A --> N[🔍 Shared Understanding]
    B --> N
    C --> N
    D --> N

• 💸 Financial Stress Composite - Debt + delinquency + evictions + stress index
• 🚨 Crisis Alert Composite - Overdoses + suicides + long-term unemployment
• 🤝 Community Health Composite - Living alone + commute + digital divide (inverted)
• 🆓 Worker Agency Index - Quit rate + job openings / unemployment
• 🌫️ Environmental Health Index - PM2.5 + ozone (inverted)

→ Wellbeing Implementation Guide | → Data Directory

Substrate provides the pieces. You and your AI create the connections.

Everything is designed for dual consumption:

• ✅ Human-readable - Markdown and CSV anyone can open
• ✅ Machine-parseable - Consistent formats AI can query
• ✅ Fully documented - Complete methodology for every dataset
• ✅ Linked with IDs - Unambiguous entity references

All datasets are available as CSV and Markdown files you can browse directly:

# Clone the repository
git clone https://github.com/danielmiessler/Substrate.git
cd Substrate

# Explore core datasets
ls Data/*/

# Explore wellbeing data sources
ls Data-Sources/*/

# Install dependencies
bun install

# Update a specific dataset
cd Data/US-GDP
bun run update.ts

# Update a wellbeing source (requires API key)
export FRED_API_KEY="your_key_here"
cd Data-Sources/DS-00004—FRED_Economic_Wellbeing
bun run update.ts

# Update all datasets
bun run scripts/update-all.ts

→ Complete Getting Started Guide | → Quick Reference

1. Fork the repository on GitHub
2. Add your component in the appropriate directory (Problems/, Solutions/, etc.)
3. Follow the format in that directory's README
4. Submit a Pull Request

→ Contribution Guidelines

Phase 4: Enhanced Access (Q4 2025 - Q1 2026)

• 🎨 Web-based contribution interface (no coding required)
• 📊 Interactive data visualizations
• 🔌 RESTful API for programmatic access
• 📱 Mobile-friendly exploration

Phase 5: Dataset Expansion (2026)

• 🌍 Additional international sources (UNICEF, OECD, IHME)
• ⚡ Real-time data feeds integration
• 🗳️ Community-driven dataset requests
• 🤝 Partnerships with research institutions

Phase 6: Intelligence Layer (2026+)

• 🤖 Automated relationship discovery
• 📈 Confidence scoring for AI-suggested links
• 🎯 Pattern detection algorithms
• 🔔 Email/Slack notifications for data updates
• 📚 Machine-readable catalog (DCAT/CKAN compliance)

Substrate provides evidence. TELOS provides intention.

Together: Complete system for intention + evidence-based action.

Daniel Miessler • July 2024

Inspiration & Contributions:

• Jonathan Dunn @xssdoctor - Similar vision and collaboration
• Joel Parish - Structure wisdom and guidance
• Joseph Thacker - Continuous flow of innovative ideas

Community Contributors:

@ThatNateGuy • @JaymanW • @karai114 • @DesertEaglePWN • @ktfth

MIT License - see the LICENSE file for details.