Data Sources

Where the data comes from, what it covers, and what it does not. Last updated: May 2026

← Back to shibui.finance

Prices, statements, and valuations

A licensed financial data provider supplies daily prices, financial statements, and valuation metrics. Data is loaded into the database daily via API.

Dataset History Notes
Daily prices (OHLCV) 1962 - present ~31M rows. Split-adjusted. All NYSE + NASDAQ listed securities.
Financial statements 1990 - present Quarterly and annual. Income statement, balance sheet, cash flow. 70+ line items per filing.
Valuation metrics 1993 - present Daily: market cap, enterprise value, P/E, P/B, trailing PEG.
Derived daily ratios 1993 - present Computed from prices and statements: EV/EBIT, EV/EBITDA, P/S, P/CF, FCF yield, dividend yield, payout ratio, earnings yield.
Derived quarterly ratios 1990 - present Computed from statements: asset turnover, EBITDA margin, ROIC, working capital, ROE TTM, profit margin TTM.
Technical indicators 1962 - present 56 indicators computed from prices. Includes RSI, MACD, Bollinger Bands, SMA, EMA, ADX, ATR, Stochastic, CCI, OBV, 15 candlestick patterns, and more.
Company metadata Current Sector/industry (Morningstar classification), SIC codes, company website, reporting currency.

Analyst data, classification, and ownership

A second licensed data provider supplements fields not covered by the primary source, including GICS classification, analyst estimates, earnings history, and ownership statistics.

Dataset Coverage Notes
GICS classification Current Sector, industry group, industry, sub-industry. Primary source for GICS taxonomy.
Interest income Sparse pre-2019, ~60% from 2019+ Interest income and net interest income. Not available from the primary source.
Analyst estimates Current snapshot Wall Street target prices, consensus EPS estimates (current and next year/quarter).
Earnings history Quarterly EPS actuals, estimates, surprise %, before/after market timing.
Ownership statistics Current snapshot Insider %, institutional %, short interest %, shares float.
Company details Current Business descriptions, HQ location, employee count, IPO date, CIK, ISIN.

Classification and listings

How GICS sectors are assigned

GICS (Global Industry Classification Standard) sector and industry codes are assigned through a three-tier fallback process:

  1. Direct GICS (primary): GICS sector, industry group, industry, and sub-industry from the supplementary data provider.
  2. Morningstar-to-GICS mapping (fallback): The primary data provider supplies Morningstar sector and industry classifications. A 166-entry mapping table converts these to the GICS hierarchy. Used when direct GICS data is missing.
  3. Nasdaq-to-GICS mapping (last resort): Nasdaq screener sector mapped to GICS sector only (no industry or sub-industry resolution). 13-entry mapping. Used when both direct GICS and Morningstar data are unavailable.

Approximately 4,150 securities (mostly ETFs, closed-end funds, and preferred stocks) lack GICS classification entirely.

Optional filters

AAOIFI Shariah compliance filter: A static exclusion list based on MSCI GICS-level screening rules. Excludes sectors and sub-industries incompatible with Islamic finance principles (conventional banking, alcohol, gambling, tobacco, weapons, entertainment). 13 exclusion rules mapped to GICS sector or sub-industry codes. Available as a reference table for filtering; not applied by default.

Coverage

Limitations

  • No intraday or real-time data
  • No options, futures, forex, or commodities
  • No international markets
  • Analyst estimates and ownership data are current snapshots, not historical time series
  • Interest income coverage is incomplete before 2019
  • ~4,150 securities lack GICS sector classification (mostly ETFs and funds)
  • Prices are split-adjusted but not dividend-adjusted
  • Data is sourced from third-party providers. We do not guarantee that any data is complete, current, or error-free
  • This is a data tool, not financial advice. See Terms of Service for full legal terms

Contact

Questions about data coverage or accuracy? chris@shibui.finance