Overview

What You'll Learn

What You'll Build

What You'll Need

Source Code

Why RAG Falls Short

Top-K Ceiling

Semantic Only

No Compute

Why Agentic Search Isn't Enough Either

Why Bigger Context Windows Don't Solve This

Why Coding Agents Fall Short

Query Types That Break Traditional RAG

Traditional RAG vs Analytical Search

Answer: "Based on the provided documents, several companies mention cybersecurity risks, including Boeing and GE. However, there may be more that were not analyzed."

Answer: "5 filings filed on Feb 3, 2025 mention substantive cybersecurity risks. The companies are: [complete list with citations]."

How Analytical Search Works

Layer 1: Search to Prune

Cost note: Adaptive depth optimizes spend in both directions. Better coverage reduces wasted downstream AI function calls that would produce wrong answers. Better restraint avoids paying to extract and classify documents that would not have answered the question in the first place. AI functions are powerful but not free — every unnecessary call adds latency and dollars. Adaptive depth is the layer that decides how many of those calls are worth making.

Layer 2: AI Functions and SQL to Analyze

Auto-Routing

Planning Mode

Environment Setup

File: sql/00_env_setup.sql

USE ROLE ACCOUNTADMIN;

-- Configuration — edit these values
SET config_database   = 'SEC_FILINGS';
SET config_schema     = 'FILING_DATA';
SET config_warehouse  = 'FILING_WH';
SET config_user_agent = 'YourOrg SEC-Filing-Demo your_name@company.com';
SET config_start_date = '2025-02-03';  -- single day for quickstart
SET config_end_date   = '2025-02-03';

-- Create database and schema
CREATE DATABASE IF NOT EXISTS IDENTIFIER($config_database);
USE DATABASE IDENTIFIER($config_database);
CREATE SCHEMA IF NOT EXISTS IDENTIFIER($config_schema);
USE SCHEMA IDENTIFIER($config_schema);

-- Create a dedicated warehouse
CREATE WAREHOUSE IF NOT EXISTS IDENTIFIER($config_warehouse)
    WAREHOUSE_SIZE = 'SMALL'
    AUTO_SUSPEND = 60
    AUTO_RESUME = TRUE
    INITIALLY_SUSPENDED = TRUE
    COMMENT = 'SEC pipeline: dynamically resized by RUN_PIPELINE()';
USE WAREHOUSE IDENTIFIER($config_warehouse);

-- Network access for SEC EDGAR
CREATE OR REPLACE NETWORK RULE SEC_EDGAR_NETWORK_RULE
    MODE = EGRESS
    TYPE = HOST_PORT
    VALUE_LIST = ('www.sec.gov:443', 'data.sec.gov:443', 'efts.sec.gov:443');

CREATE OR REPLACE EXTERNAL ACCESS INTEGRATION SEC_EDGAR_EAI
    ALLOWED_NETWORK_RULES = (SEC_EDGAR_NETWORK_RULE)
    ENABLED = TRUE;

NOTE: SEC EDGAR requires a valid User-Agent header with your organization name and contact email. Edit config_user_agent above — this value is passed as a parameter to RUN_PIPELINE(), which forwards it to all HTTP calls. You only need to set it in this one place. See SEC EDGAR access policies for details.

Build the Data Pipeline

File: sql/01_pipeline.sql

Core Tables

Running the Pipeline

-- The last command in the script calls the pipeline:
CALL RUN_PIPELINE('2025-02-03', '2025-02-03', 'YourOrg SEC-Filing-Demo your_name@company.com');

NOTE: The pipeline dynamically resizes the warehouse between phases. Total runtime for a single day: ~3-5 minutes.

Scaling Beyond One Day

This quickstart ingests a single day for speed (~3-5 minutes). To load more historical data, simply widen config_start_date and config_end_date in 00_env_setup.sql — the pipeline handles any date range up to a full year (runtime scales at ~3-5 minutes per business day). For large backfills (e.g., a full year), you can parallelize by creating multiple Snowflake Tasks that each call RUN_PIPELINE() with non-overlapping monthly date ranges (e.g., Jan, Feb, Mar…), allowing months to process concurrently.

For continuous automated ingestion, schedule a single Snowflake Task with a daily CRON to call RUN_PIPELINE() for the current day — new filings flow into the Cortex Search service and Semantic View automatically with no manual intervention.

Verify the Pipeline Output

-- Check corpus statistics
SELECT
    COUNT(*)                       AS total_chunks,
    COUNT(DISTINCT ACCESSION_NO)   AS distinct_filings,
    COUNT(DISTINCT TICKER)         AS distinct_tickers,
    COUNT(DISTINCT FORM_TYPE)      AS distinct_form_types,
    MIN(FILED_AT)                  AS earliest_filing,
    MAX(FILED_AT)                  AS latest_filing,
    AVG(LENGTH(CHUNK_TEXT))::INT   AS avg_chunk_chars
FROM FILING_CHUNKS
WHERE CHUNK_TEXT IS NOT NULL;

Create the Cortex Search Service

File: sql/02_create_search_service.sql

NOTE: If you already have a Cortex Search service, you can reuse it; you don't need to create a new service specifically for analytical search. If your agent already has a Cortex Search tool configured, you don't need to add another one.

Create the Service

USE ROLE ACCOUNTADMIN;
USE WAREHOUSE FILING_WH;

CREATE OR REPLACE CORTEX SEARCH SERVICE SEC_FILINGS.FILING_DATA.SEC_FILING_SEARCH
    TEXT INDEXES CHUNK_TEXT, CHUNK_ID, ACCESSION_NO, SECTION_NAME, COMPANY_NAME
    VECTOR INDEXES CHUNK_TEXT (model='snowflake-arctic-embed-l-v2.0')
    ATTRIBUTES COMPANY_NAME, TICKER, FORM_TYPE, SECTION_NAME, FILED_AT,
               PERIOD_OF_REPORT, INDUSTRY_SECTOR, INDUSTRY_TITLE, CHUNK_ID, ACCESSION_NO
    WAREHOUSE = FILING_WH
    TARGET_LAG = '1 day'
    COMMENT = 'SEC filing search - multi-index with text and vector'
AS (
    SELECT
        CHUNK_ID, CHUNK_TEXT, ACCESSION_NO, COMPANY_NAME, TICKER, FORM_TYPE,
        SECTION_NAME,
        TO_VARCHAR(FILED_AT, 'YYYY-MM-DD') AS FILED_AT,
        TO_VARCHAR(PERIOD_OF_REPORT, 'YYYY-MM-DD') AS PERIOD_OF_REPORT,
        COALESCE(INDUSTRY_SECTOR, 'Other') AS INDUSTRY_SECTOR,
        INDUSTRY_TITLE
    FROM SEC_FILINGS.FILING_DATA.FILING_CHUNKS
    WHERE CHUNK_TEXT IS NOT NULL AND LENGTH(CHUNK_TEXT) > 100
);

Why Each Design Choice Matters for Analytical Search

NOTE: The search service indexes ~3,283 of the 3,453 total chunks. The WHERE LENGTH(CHUNK_TEXT) > 100 filter excludes very short chunks (section headers, boilerplate) that would add noise to search results.

Create the Semantic View

File: sql/03_create_semantic_view.sql

Create the Semantic View

USE ROLE ACCOUNTADMIN;
USE DATABASE SEC_FILINGS;
USE SCHEMA FILING_DATA;
USE WAREHOUSE FILING_WH;

CREATE OR REPLACE SEMANTIC VIEW SEC_FILING_ANALYTICS
  TABLES (
    signals AS FILING_SIGNALS
      PRIMARY KEY (SIGNAL_ID)
      WITH SYNONYMS = ('investment signals', 'filing signals', 'EDGAR signals', 'SEC filings')
      COMMENT = 'AI-extracted investment signals from SEC EDGAR filings.',
    meta AS FILING_INDEX
      PRIMARY KEY (ACCESSION_NO)
      WITH SYNONYMS = ('filing metadata', 'EDGAR index', 'filing registry')
      COMMENT = 'SEC EDGAR filing metadata: accession numbers, CIKs, filing URLs, dates'
  )
  RELATIONSHIPS (
    signals_to_meta AS signals(ACCESSION_NO) REFERENCES meta(ACCESSION_NO)
  )
  FACTS (
    signals.accession_no AS signals.ACCESSION_NO
      WITH SYNONYMS = ('accession number', 'filing id')
      COMMENT = 'EDGAR accession number uniquely identifying the filing',
    signals.revenue AS signals.REVENUE
      WITH SYNONYMS = ('total revenue', 'sales', 'top line')
      COMMENT = 'Revenue in millions USD. NULL if not extractable.',
    signals.net_income AS signals.NET_INCOME
      WITH SYNONYMS = ('net income', 'profit', 'bottom line')
      COMMENT = 'Net income figure extracted from filing.',
    signals.eps AS signals.EPS
      WITH SYNONYMS = ('earnings per share', 'diluted EPS')
      COMMENT = 'Normalized EPS value.',
    signals.yoy_change AS signals.YOY_CHANGE
      WITH SYNONYMS = ('year over year', 'YoY growth', 'growth rate')
      COMMENT = 'Year-over-year change percentage.',
    signals.forward_guidance AS signals.FORWARD_GUIDANCE
      WITH SYNONYMS = ('guidance', 'outlook', 'forecast')
      COMMENT = 'Forward-looking financial guidance from MD&A.'
  )
  DIMENSIONS (
    signals.company_name AS signals.COMPANY_NAME
      WITH SYNONYMS = ('company', 'filer', 'issuer')
      COMMENT = 'Company that filed the SEC document',
    signals.ticker AS signals.TICKER
      WITH SYNONYMS = ('stock ticker', 'symbol')
      COMMENT = 'Stock ticker symbol. May be NULL for non-public filers.',
    signals.form_type AS signals.FORM_TYPE
      WITH SYNONYMS = ('filing type', 'SEC form')
      COMMENT = '10-K (annual), 10-Q (quarterly), 8-K (current report)',
    signals.event_type AS COALESCE(signals.EVENT_TYPE_NORMALIZED, signals.EVENT_TYPE)
      WITH SYNONYMS = ('event', 'signal type', 'event classification')
      COMMENT = 'AI-classified event type: Earnings, M&A, Leadership Change, Risk Disclosure, Guidance Update, Regulatory, Capital Markets, Bankruptcy, Annual Report, Quarterly Report, Current Report, Other.',
    signals.sentiment AS signals.SENTIMENT
      WITH SYNONYMS = ('tone', 'filing sentiment')
      COMMENT = 'AI-assessed sentiment: POSITIVE, NEGATIVE, NEUTRAL, MIXED.',
    signals.industry_sector AS COALESCE(signals.INDUSTRY_SECTOR, 'Other')
      WITH SYNONYMS = ('sector', 'industry')
      COMMENT = 'SEC Office-based industry sector: Technology, Life Sciences, Finance, Real Estate & Construction, Energy & Transportation, Manufacturing, Trade & Services, Other.',
    signals.industry_title AS signals.INDUSTRY_TITLE
      WITH SYNONYMS = ('specific industry', 'sub-sector')
      COMMENT = 'Specific SEC industry title.',
    signals.is_amendment AS signals.IS_AMENDMENT
      WITH SYNONYMS = ('amendment', 'restated')
      COMMENT = 'TRUE if this is an amended filing.',
    meta.cik AS meta.CIK
      WITH SYNONYMS = ('SEC CIK', 'central index key')
      COMMENT = 'SEC Central Index Key',
    signals.signal_date AS signals.SIGNAL_DATE
      WITH SYNONYMS = ('filing date', 'date filed', 'when filed')
      COMMENT = 'The date the SEC received the filing.',
    signals.period_of_report AS signals.PERIOD_OF_REPORT
      WITH SYNONYMS = ('fiscal period', 'report period', 'period end')
      COMMENT = 'Fiscal period end date the filing covers.'
  )
  METRICS (
    signals.filing_count AS COUNT(signals.SIGNAL_ID)
      WITH SYNONYMS = ('number of filings', 'total filings', 'how many filings')
      COMMENT = 'Total number of filings matching filters',
    signals.positive_signals AS COUNT(CASE WHEN signals.SENTIMENT = 'POSITIVE' THEN 1 END)
      WITH SYNONYMS = ('positive filings', 'bullish signals')
      COMMENT = 'Count of filings with positive sentiment',
    signals.negative_signals AS COUNT(CASE WHEN signals.SENTIMENT = 'NEGATIVE' THEN 1 END)
      WITH SYNONYMS = ('negative filings', 'bearish signals')
      COMMENT = 'Count of filings with negative sentiment',
    signals.ma_count AS COUNT(CASE WHEN signals.EVENT_TYPE = 'M&A' THEN 1 END)
      WITH SYNONYMS = ('merger filings', 'M&A events', 'deals')
      COMMENT = 'Count of merger and acquisition events',
    signals.leadership_change_count AS COUNT(CASE WHEN signals.EVENT_TYPE = 'Leadership Change' THEN 1 END)
      WITH SYNONYMS = ('leadership events', 'management changes')
      COMMENT = 'Count of leadership change events',
    signals.negative_sentiment_pct AS
      ROUND(100.0 * COUNT(CASE WHEN signals.SENTIMENT = 'NEGATIVE' THEN 1 END)
            / NULLIF(COUNT(signals.SIGNAL_ID), 0), 2)
      WITH SYNONYMS = ('negative rate', 'percent negative')
      COMMENT = 'Percentage of filings with negative sentiment (0-100 scale)'
  )
  COMMENT = 'Investment signal analytics over SEC EDGAR filing corpus.'
  AI_SQL_GENERATION 'This semantic view covers SEC EDGAR filings. SIGNAL_DATE is the authoritative filing timestamp. EVENT_TYPE and SENTIMENT are AI-extracted.';

Deploy the Analytical Search Agent

File: sql/04_deploy_agent.sql

Create the Agent

USE ROLE ACCOUNTADMIN;
USE DATABASE SEC_FILINGS;
USE SCHEMA FILING_DATA;

-- Ensure Snowflake Intelligence (CoWork) object exists
CREATE SNOWFLAKE INTELLIGENCE IF NOT EXISTS SNOWFLAKE_INTELLIGENCE_OBJECT_DEFAULT;

-- Deploy the Analytical Search agent (co-located with search service + semantic view)
CREATE OR REPLACE AGENT SEC_FILINGS.FILING_DATA.SEC_ANALYTICAL_SEARCH_AGENT
COMMENT = 'SEC filing research agent v7 - FILED_AT/PERIOD_OF_REPORT are now native DATE in the search service, range filters supported.'
FROM SPECIFICATION $$
{
  "models": { "orchestration": "claude-opus-4-7" },
  "orchestration": { "budget": { "seconds": 600, "tokens": 200000 } },
  "instructions": {
    "orchestration": "You are SEC Filing Analyst, an investment-research agent over a corpus of SEC EDGAR 10-K, 10-Q, 8-K filings.\n\nFor questions about counts, comparisons, exhaustive lists, or cross-filing patterns:\n1. Search broadly using multiple relevant queries — use synonyms and related phrasings.\n2. Return a clear answer with company name, form type, filing date, and the specific evidence from each filing chunk."
  },
  "tools": [
    {
      "tool_spec": {
        "type": "cortex_search",
        "name": "filing_semantic_search",
        "description": "Multi-index Cortex Search over SEC filing chunks."
      }
    },
    {
      "tool_spec": {
        "type": "cortex_analyst_text_to_sql",
        "name": "filing_analyst",
        "description": "Structured analytics over pre-extracted filing signals (sector counts, sentiment percentages, signal trends)."
      }
    }
  ],
  "tool_resources": {
    "filing_semantic_search": {
      "name": "SEC_FILINGS.FILING_DATA.SEC_FILING_SEARCH",
      "search_service": "SEC_FILINGS.FILING_DATA.SEC_FILING_SEARCH",
      "database_schema": "SEC_FILINGS.FILING_DATA",
      "is_multi_index": true,
      "max_results": 1000,
      "id_column": "CHUNK_ID",
      "title_column": "COMPANY_NAME",
      "base_table": "SEC_FILINGS.FILING_DATA.FILING_CHUNKS",
      "base_table_columns": [
        "CHUNK_ID","CHUNK_TEXT","ACCESSION_NO","COMPANY_NAME","TICKER",
        "FORM_TYPE","SECTION_NAME","FILED_AT","PERIOD_OF_REPORT",
        "INDUSTRY_SECTOR","INDUSTRY_TITLE"
      ],
      "execution_environment": {"type": "warehouse", "warehouse": "FILING_WH"},
      "columns_and_descriptions": {
        "CHUNK_ID":         {"description": "Chunk primary key", "type": "string", "searchable": true,  "filterable": true},
        "CHUNK_TEXT":       {"description": "Full filing passage text. Search here for filing content.", "type": "string", "searchable": true, "filterable": false},
        "COMPANY_NAME":     {"description": "Filer company name. Search here for specific companies.", "type": "string", "searchable": true, "filterable": true},
        "TICKER":           {"description": "Stock ticker (e.g. NVDA, MSFT, GOOGL). Use for searching or filtering filings by specific public companies. Approximately 85% of filings have a populated ticker.", "type": "string", "searchable": true, "filterable": true},
        "FORM_TYPE":        {"description": "10-K, 10-K/A, 10-Q, 10-Q/A, 8-K, 8-K/A.", "type": "string", "searchable": false, "filterable": true},
        "SECTION_NAME":     {"description": "Filing section (Risk Factors, MD&A, Item 1.01, etc.).", "type": "string", "searchable": true, "filterable": true},
        "FILED_AT":         {"description": "SEC filing date (DATE). Supports @gte / @lte range filters.", "type": "date", "searchable": false, "filterable": true},
        "PERIOD_OF_REPORT": {"description": "Fiscal period end date (DATE). Supports @gte / @lte range filters.", "type": "date", "searchable": false, "filterable": true},
        "INDUSTRY_SECTOR":  {"description": "SIC-based sector grouping.", "type": "string", "searchable": false, "filterable": true},
        "INDUSTRY_TITLE":   {"description": "Detailed SIC industry title.", "type": "string", "searchable": false, "filterable": true},
        "ACCESSION_NO":     {"description": "SEC accession number, unique per filing.", "type": "string", "searchable": false, "filterable": true}
      }
    },
    "filing_analyst": {
      "semantic_view": "SEC_FILINGS.FILING_DATA.SEC_FILING_ANALYTICS",
      "execution_environment": {"type": "warehouse", "warehouse": "FILING_WH"}
    }
  }
}
$$;

-- Register in Snowflake CoWork
ALTER SNOWFLAKE INTELLIGENCE SNOWFLAKE_INTELLIGENCE_OBJECT_DEFAULT
    ADD AGENT SEC_FILINGS.FILING_DATA.SEC_ANALYTICAL_SEARCH_AGENT;

Tools and Their Roles

NOTE: Column descriptions are the single most impactful thing you can do to improve analytical search quality. The agent uses them to decide which columns to filter on, how to interpret values, and how to frame AI_FILTER and AI_EXTRACT calls. For each column, describe:

• What it contains and its expected format or value range
• Sample values or enumerations (e.g., "Values: 10-K, 10-Q, 8-K, 8-K/A")
• Whether it's suitable for filtering, searching, or extraction
• Any relationships to other columns

Columns without descriptions are harder for the agent to use effectively, especially filterable attributes that determine how the search is scoped.

NOTE: Snowflake recommends setting max_results to 1,000 for analytical search. This gives the agent enough breadth to surface the full relevant set of documents. Adaptive depth limits actual compute to what the question requires — you won't pay for 1,000 AI function calls on a narrow question.

Analytical Search in Action

Performance expectations: Simple RAG questions typically complete in ~30 seconds. Analytical search workflows take 2–6 minutes for most questions. Complex analyses over large corpora (thousands of documents with multi-step extraction and aggregation) may take up to 15 minutes. This is expected — the agent is doing real computation, not just summarizing a handful of passages.

Cost considerations: Analytical search incurs costs from agent orchestration and the AI functions it uses (AI_FILTER, AI_EXTRACT, AI_AGG). Each AI function call processes one row. Adaptive depth limits unnecessary calls by stopping retrieval when results are no longer relevant — so you pay only for the documents that matter to the answer. For detailed AI function pricing, see the Cortex AI cost documentation.

Exercise 1: Auto-Routing (RAG Path)

What does Boeing's 10-K say about supply chain risk?

---

Exercise 2: Counting (Analytical Search Path)

How many filings filed on Feb 3, 2025 mention cybersecurity risks?

---

Exercise 3: Exhaustive Listing with Extraction

List every company that announced a leadership change in an 8-K filing on Feb 3, 2025.

---

Exercise 4: Honest Refusal

Compare Apple and Microsoft 10-K risk factor language about AI — what specific concerns does each company raise?

Compare Boeing and RTX 10-K risk factor language — what does each emphasize?

---

Bonus Exercises

Cleanup

File: sql/99_teardown.sql

USE ROLE ACCOUNTADMIN;

-- Drop database (removes all tables, views, UDFs, procedures)
DROP DATABASE IF EXISTS SEC_FILINGS;

-- Drop warehouse
DROP WAREHOUSE IF EXISTS FILING_WH;

-- Drop external access integration
DROP INTEGRATION IF EXISTS SEC_EDGAR_EAI;

Conclusion And Resources

What You Built

From Quickstart to Production

Key Takeaways

Resources