Skip to content

Getting started

This guide takes you from an empty directory to a searchable index of your own site. You will need Python 3.13 and MySQL 8 or later.

What you need

  • Python 3.13 — from python.org or your package manager.
  • MySQL 8+ — MySQL Community Edition is free. MariaDB 11 works too; the schema uses nothing exotic.
  • Git — to clone the source.

A shared hosting account is enough. Dead Simple Search needs no MySQL plugins and no server-configuration changes.

Older Python versions

The project targets Python 3.13. Earlier versions may work but are not tested.

Step 1: Get the code

git clone https://codeberg.org/marcusosterberg/Dead-Simple-Search.git
cd Dead-Simple-Search

Step 2: Set up Python

Using uv:

# Install uv if you don't have it — see https://docs.astral.sh/uv/
curl -LsSf https://astral.sh/uv/install.sh | sh

uv venv --python 3.13
source .venv/bin/activate
uv pip install -r requirements.txt

Or with the standard library's venv:

python3.13 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Install the dependencies, don't skip them

protego handles wildcard rules in robots.txt. Without it the application still runs, but falls back to a parser that ignores those rules and logs a warning — meaning it may crawl paths a site asked it to leave alone.

Step 3: Set up MySQL

Create a database and a user:

CREATE DATABASE IF NOT EXISTS deadsimplesearch
  CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

CREATE USER IF NOT EXISTS 'deadsimplesearch'@'localhost'
  IDENTIFIED BY 'choose-a-strong-password';

GRANT ALL PRIVILEGES ON deadsimplesearch.*
  TO 'deadsimplesearch'@'localhost';

FLUSH PRIVILEGES;

That is all the manual setup required. The tables are created automatically on the first API request, and later versions add their own columns the same way — there is no separate migration step and no schema file to import.

Step 4: Configure

Set your database password and an API key:

export MYSQL_PASSWORD=choose-a-strong-password
export API_KEY=$(python -c "import secrets; print(secrets.token_urlsafe(32))")

Without API_KEY, the endpoints that register sites and trigger crawls accept unauthenticated requests. That is fine on your laptop and a bad idea anywhere else.

If exporting variables is inconvenient on your host, put the same KEY=VALUE lines in a .env file beside config.py.

See the configuration reference for every available setting.

Step 5: Run

python app.py

The server listens on http://localhost:5555. For production, run it under gunicorn with a single worker:

gunicorn --workers 1 --bind 0.0.0.0:5555 wsgi:app

Step 6: Add your first site

curl -X POST http://localhost:5555/api/sites \
  -H "X-API-Key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"domain": "yourwebsite.com", "start_url": "https://yourwebsite.com/"}'

Trigger a crawl:

curl -X POST http://localhost:5555/api/sites/1/crawl \
  -H "X-API-Key: $API_KEY"

The crawl runs in the background. Check on it:

curl http://localhost:5555/api/sites/1/crawl/status

Once it reports completed, search:

curl "http://localhost:5555/api/sites/1/search?q=your+search+terms"

Search and status are read endpoints and need no key.

Getting good dates in your results

Results carry page_published and page_modified. Both come first from the page's own markup, so a site that declares its dates gets the best data. Adding Schema.org JSON-LD to your templates is the single most useful thing you can do for search result quality:

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "WebPage",
  "datePublished": "2019-03-04",
  "dateModified": "2026-01-15"
}
</script>

Open Graph, Microdata and Dublin Core work too. Without any of them, page_modified can still fall back to your sitemap's <lastmod> or the HTTP Last-Modified header, but page_published will be empty — no external source can know when a page was created.

Next steps