๐ŸŒ AIๆœ็ดข & ไปฃ็† ไธป้กต
Skip to content
/ databaseSnake Public

databaseSnake is a fast, type-safe Python file-based search engine that lets you quickly query CSV, TXT or SQL dump files with memory-mapped I/O and parallel search.

0 stars 0 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

xsyncio/databaseSnake

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

3 Commits
.vscode
.vscode
ย 
ย 
dbsearcher
dbsearcher
ย 
ย 
.gitignore
.gitignore
ย 
ย 
README.md
README.md
ย 
ย 
pyproject.toml
pyproject.toml
ย 
ย 

Repository files navigation

databaseSnake

Typing SVG

Stars Issues Python

Author Type Safe Version 

   โ–„โ–„                  โ–„โ–„                       โ–„โ–„โ–„โ–„โ–„โ–„โ–„                          
   โ–ˆโ–ˆ        โ–ˆโ–ˆ        โ–ˆโ–ˆ                      โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–€โ–€โ–€             โ–„โ–„           
โ–„โ–ˆโ–ˆโ–ˆโ–ˆ  โ–€โ–€โ–ˆโ–„ โ–€โ–ˆโ–ˆโ–€โ–€ โ–€โ–€โ–ˆโ–„ โ–ˆโ–ˆโ–ˆโ–ˆโ–„  โ–€โ–€โ–ˆโ–„ โ–„โ–ˆโ–€โ–€โ–€ โ–„โ–ˆโ–€โ–ˆโ–„  โ–€โ–ˆโ–ˆโ–ˆโ–ˆโ–„  โ–ˆโ–ˆโ–ˆโ–ˆโ–„  โ–€โ–€โ–ˆโ–„ โ–ˆโ–ˆ โ–„โ–ˆโ–€ โ–„โ–ˆโ–€โ–ˆโ–„ 
โ–ˆโ–ˆ โ–ˆโ–ˆ โ–„โ–ˆโ–€โ–ˆโ–ˆ  โ–ˆโ–ˆ  โ–„โ–ˆโ–€โ–ˆโ–ˆ โ–ˆโ–ˆ โ–ˆโ–ˆ โ–„โ–ˆโ–€โ–ˆโ–ˆ โ–€โ–ˆโ–ˆโ–ˆโ–„ โ–ˆโ–ˆโ–„โ–ˆโ–€    โ–€โ–ˆโ–ˆโ–ˆโ–ˆ โ–ˆโ–ˆ โ–ˆโ–ˆ โ–„โ–ˆโ–€โ–ˆโ–ˆ โ–ˆโ–ˆโ–ˆโ–ˆ   โ–ˆโ–ˆโ–„โ–ˆโ–€ 
โ–€โ–ˆโ–ˆโ–ˆโ–ˆ โ–€โ–ˆโ–„โ–ˆโ–ˆ  โ–ˆโ–ˆ  โ–€โ–ˆโ–„โ–ˆโ–ˆ โ–ˆโ–ˆโ–ˆโ–ˆโ–€ โ–€โ–ˆโ–„โ–ˆโ–ˆ โ–„โ–„โ–„โ–ˆโ–€ โ–€โ–ˆโ–„โ–„โ–„ โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–€ โ–ˆโ–ˆ โ–ˆโ–ˆ โ–€โ–ˆโ–„โ–ˆโ–ˆ โ–ˆโ–ˆ โ–€โ–ˆโ–„ โ–€โ–ˆโ–„โ–„โ–„

๐Ÿš€ Features

โšก Blazing Fast

  • Memory-mapped I/O for files >10MB
  • Parallel search with ThreadPoolExecutor
  • LRU caching for repeated queries
  • Optimized casefold() for case-insensitive search

๐Ÿ”’ Type Perfect

  • basedpyright at strictest settings
  • Zero errors, zero warnings
  • Full type annotations everywhere
  • No # type: ignore anywhere

๐Ÿ“ Multi-Format Support

  • CSV files (comma-separated)
  • TXT files (line-by-line)
  • SQL files (data dumps)
  • Automatic parser selection

๐ŸŽจ Beautiful Console

  • Rich console output with colors
  • Progress animations
  • Styled search results
  • TTY-safe color detection

๐Ÿ“ฆ Installation

# Clone the repository
git clone https://github.com/xsyncio/databaseSnake.git
cd databaseSnake

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # Linux/Mac
# .venv\Scripts\activate   # Windows

# Install dependencies
pip install -r requirements.txt
# Or install as package
pip install -e .

๐Ÿ Quick Start

# Run the application
python main.py

# Or run as module
python -m dbsearcher
๐Ÿ“ธ Screenshot Preview 
โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚    โ–„โ–„                  โ–„โ–„                       โ–„โ–„โ–„โ–„โ–„โ–„โ–„                    โ”‚
โ”‚    โ–ˆโ–ˆ        โ–ˆโ–ˆ        โ–ˆโ–ˆ                      โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–€โ–€โ–€             โ–„โ–„     โ”‚
โ”‚ โ–„โ–ˆโ–ˆโ–ˆโ–ˆ  โ–€โ–€โ–ˆโ–„ โ–€โ–ˆโ–ˆโ–€โ–€ โ–€โ–€โ–ˆโ–„ โ–ˆโ–ˆโ–ˆโ–ˆโ–„  โ–€โ–€โ–ˆโ–„ โ–„โ–ˆโ–€โ–€โ–€ โ–„โ–ˆโ–€โ–ˆโ–„  โ–€โ–ˆโ–ˆโ–ˆโ–ˆโ–„  โ–ˆโ–ˆโ–ˆโ–ˆโ–„  โ–€โ–€โ–ˆโ–„ โ–ˆโ–ˆ โ–„โ–ˆโ–€ โ”‚
โ”‚ โ–ˆโ–ˆ โ–ˆโ–ˆ โ–„โ–ˆโ–€โ–ˆโ–ˆ  โ–ˆโ–ˆ  โ–„โ–ˆโ–€โ–ˆโ–ˆ โ–ˆโ–ˆ โ–ˆโ–ˆ โ–„โ–ˆโ–€โ–ˆโ–ˆ โ–€โ–ˆโ–ˆโ–ˆโ–„ โ–ˆโ–ˆโ–„โ–ˆโ–€    โ–€โ–ˆโ–ˆโ–ˆโ–ˆ โ–ˆโ–ˆ โ–ˆโ–ˆ โ–„โ–ˆโ–€โ–ˆโ–ˆ โ–ˆโ–ˆโ–ˆโ–ˆ  โ”‚
โ”‚ โ–€โ–ˆโ–ˆโ–ˆโ–ˆ โ–€โ–ˆโ–„โ–ˆโ–ˆ  โ–ˆโ–ˆ  โ–€โ–ˆโ–„โ–ˆโ–ˆ โ–ˆโ–ˆโ–ˆโ–ˆโ–€ โ–€โ–ˆโ–„โ–ˆโ–ˆ โ–„โ–„โ–„โ–ˆโ–€ โ–€โ–ˆโ–„โ–„โ–„ โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–€ โ–ˆโ–ˆ โ–ˆโ–ˆ โ–€โ–ˆโ–„โ–ˆโ–ˆ โ–ˆโ–ˆ โ–€โ–ˆโ–„โ”‚
โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ
โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚ v1.0.0 | by Xsyncio | https://github.com/xsyncio     โ”‚
โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ

๐Ÿ“ Files: 18 | ๐Ÿ’พ Size: 9718.45 MB

๐Ÿ” Search Examples:
โ€ข User ID: 556343434
โ€ข Phone: +19000000000
โ€ข Name: John
โ€ข Email: example@mail.eg

1. Search in databases
2. Exit

Choose an option: _

๐Ÿ—๏ธ Architecture

graph TD
    A[Terminal] --> B["__main__.py"]
    B --> C[databaseSnake]
    C --> D[SearchEngine]
    D --> E[FileIndexer]
    D --> F[ThreadPoolExecutor]
    D --> G{Parser Selection}
    G --> H[TextParser - .txt]
    G --> I[CSVParser - .csv]
    G --> J[SQLParser - .sql]
    C --> K[Rich Console UI]
    D --> L[mmap for large files]
Loading

๐Ÿ“ Project Structure

databaseSnake/
โ”œโ”€โ”€ ๐Ÿ“„ main.py                 # Entry point
โ”œโ”€โ”€ ๐Ÿ“„ pyproject.toml          # Project config
โ”œโ”€โ”€ ๐Ÿ“‚ base/                   # Your database files go here
โ”‚   โ”œโ”€โ”€ data.csv
โ”‚   โ”œโ”€โ”€ users.txt
โ”‚   โ””โ”€โ”€ dump.sql
โ””โ”€โ”€ ๐Ÿ“‚ dbsearcher/             # Main package
    โ”œโ”€โ”€ ๐Ÿ“„ __init__.py         # Package init
    โ”œโ”€โ”€ ๐Ÿ“„ __main__.py         # CLI entry
    โ”œโ”€โ”€ ๐Ÿ“„ constants.py        # Configuration
    โ”œโ”€โ”€ ๐Ÿ“„ exceptions.py       # Custom errors
    โ”œโ”€โ”€ ๐Ÿ“„ logging.py          # Rich logging
    โ”œโ”€โ”€ ๐Ÿ“„ types.py            # Type definitions
    โ”œโ”€โ”€ ๐Ÿ“‚ search/             # Search engine
    โ”‚   โ”œโ”€โ”€ engine.py          # Main coordinator
    โ”‚   โ”œโ”€โ”€ indexer.py         # File indexer
    โ”‚   โ”œโ”€โ”€ parsers.py         # Format parsers
    โ”‚   โ””โ”€โ”€ results.py         # Result formatting
    โ”œโ”€โ”€ ๐Ÿ“‚ ui/                  # Console UI
    โ”‚   โ”œโ”€โ”€ colors.py          # ANSI colors
    โ”‚   โ”œโ”€โ”€ display.py         # Display utils
    โ”‚   โ”œโ”€โ”€ effects.py         # Animations
    โ”‚   โ””โ”€โ”€ menu.py            # Menu system
    โ””โ”€โ”€ ๐Ÿ“‚ utils/              # Utilities
        โ”œโ”€โ”€ external.py        # URL opening
        โ”œโ”€โ”€ filesystem.py      # File operations
        โ””โ”€โ”€ platform.py        # Platform detection

โš™๏ธ Performance Optimizations

Technique Description Benefit
๐Ÿ—บ๏ธ mmap Memory-mapped file I/O Zero-copy reads for >10MB files
๐Ÿงต ThreadPool Parallel file processing Nร— speedup on multi-core
๐Ÿ“ฆ LRU Cache Result caching Instant repeated queries
๐Ÿ”ค casefold() Optimized case folding Faster than .lower()
โฑ๏ธ Early Exit Max results limit Stops at first N matches
๐Ÿ”„ Generators Streaming iteration Zero full-list allocation

๐Ÿ› ๏ธ Configuration

Edit dbsearcher/constants.py to customize:

# Performance tuning
MMAP_THRESHOLD_BYTES = 10 * 1024 * 1024  # Use mmap for files > 10MB
DEFAULT_PARALLEL_WORKERS = 4             # Number of parallel workers
MAX_RESULTS_DEFAULT = 10000              # Maximum results to return

# Supported file types
SUPPORTED_EXTENSIONS = (".csv", ".txt", ".sql")

๐Ÿงช Type Checking

This project uses basedpyright at the strictest settings possible:

# Install basedpyright
pip install basedpyright

# Run type checker
basedpyright dbsearcher/

# Expected output:
# 0 errors, 0 warnings, 0 notes โœ…

๐Ÿ“œ License

This project is licensed under the MIT License - see the LICENSE file for details.

๐Ÿ™ Credits

Xsyncio

Built with ๐Ÿ Python and โค๏ธ by Xsyncio

About

databaseSnake is a fast, type-safe Python file-based search engine that lets you quickly query CSV, TXT or SQL dump files with memory-mapped I/O and parallel search.

Topics

python cli search-engine database-management data-processing file-search database-library database-tools

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages