Ctrl+K

EquityUniverseFilter

EquityUniverseFilter#

class sigtech.framework.equities.equity_universe_filter.EquityUniverseFilter

Class designed to filter stocks by fundamentals, e.g. dividend yield or EBIT, by history fields, e.g. volume, VWAP, and IBES analyst estimates.

Basic filter usage:

some_dates = pd.date_range(dtm.date(2019, 1, 15), dtm.date(2020, 9, 15), freq='Q')

equity_filter = sig.EquityUniverseFilter('SPX INDEX')

# Add: top 10
equity_filter.add('MARKETCAP', 'top', 10, 'Daily')

# Add: top 50%
equity_filter.add('vwap', 'top', 50, 'Daily', percent=True)

# Add: minimum number of data points of VWAP timeseries (at least 85% non-NaNs of last 250 points, on the day
# the filter is applied)
equity_filter.add('vwap', 'min_data_size', (250, 0.85), 'Daily')

# Add: IBES analyst estimates (fields available may not coincide with the fundamental fields available for the
# the single stocks)
equity_filter.add('Price/Sales Ratio', 'top', 10, analyst_estimates=True, metric='mean_est',
    historical_period_index='FY1')

# Restriction list: some single stocks may be excluded from the universe filtering
equity_filter.exclude_stocks(['1000045.SINGLE_STOCK.TRADABLE'])

# Apply all filters previously defined:
filtered_ids = equity_filter.apply_filter(some_dates)

Examples of customised aggregation functions are shown below. The function must have one argument only, which represents either a pandas Series or a list of pandas Series. The return must be a single pandas Series representing the aggregated metric.

def compute_rolling_volume(ts):
    return ts.rolling(window=30).median()

def compute_payout_ratio(ts):
    s1, s2 = ts[0:2]
    return s1.divide(s2)

equity_filter.add('Volume', '>', 1e4, 'Daily', compute_rolling_volume)

If more fundamental fields are available, e.g. 'Dividends' and 'Net Income', then the payout ratio can be computed:

  • equity_filter.add(['Dividends', 'Net Income'], 'top', 20, 'Daily', compute_payout_ratio)

  • equity_filter.add(['Dividends', 'Net Income'], 'top', 20, ['Daily', 'Daily'], compute_payout_ratio)

The two examples of defining the frequencies corresponding to the fields used by compute_payout_ratio are equivalent. However, if a list of frequencies is provided, its length must be equal to the length of the list of fields.

In the example below, an additional stock_obj parameter is used within the aggregation function to access the single stock’s data directly. Please note that the parameters order (ts, stock_obj) cannot be changed, and additional parameters are not allowed.

def compute_rolling_volatility(ts, stock_obj):
    adjusted_prices = stock_obj.adjusted_price_history(True, True)
    return 1 / adjusted_prices.ffill().pct_change().dropna().rolling(window=21).std()

Operator precedence: order specified by the user with add().

  • Sector operators: 'in', 'not_in'.

  • Comparison operators: '<', '<=', '==', '!=', '>=', '>', 'min_data_size'.

  • Ranking operators: 'top', 'bottom', 'body'. Operator body with value x filters out the top x and bottom x stocks by the selected fundamental or history field.

To generate a composite universe from multiple stock indices:

equity_filter = sig.EquityUniverseFilter(['SPX INDEX', 'RIY INDEX'])

To retain only the most liquid listing for every stock in the universe:

from sigtech.framework.equities.liquidity_indicators import LiquidityIndicators

equity_filter = sig.EquityUniverseFilter('SPX INDEX', most_liquid=True,
                                         liquidity_function=LiquidityIndicators.rolling_dollar_vol)

Keyword argument liquidity_function is mandatory, rolling_window, min_periods_ratio and additional SIGMaster filters (filters) are optional.

To display all available operators:

equity_filter.OPERATORS

To retain all NaN values when applying comparison operators:

equity_filter = sig.EquityUniverseFilter('SPX INDEX', exclude_nans=False)

To display all available fields for a given stock identifier stock_id:

stock_id = '1000045.SINGLE_STOCK.TRADABLE'
equity_filter.available_fields(stock_id)
FREQUENCIES = ['Daily', 'Weekly', 'Monthly', 'Quarterly', 'Restated – Quarterly', 'Quarterly – Cumulative', 'Restated – Quarterly Cumulative', 'Semi-Annual', 'Annual', 'Restated – Annual', 'Trimester', 'Trimester - Cumulative', 'Restated – Semi-Annual', 'Restated – Trimester', 'Restated – Trimester Cumulative']

Fundamental frequencies available.

OPERATORS = ['<', '<=', '==', '!=', '>=', '>', 'min_data_size', 'top', 'bottom', 'body', 'in', 'not_in']

Filter operators available.

TRBC_COLUMNS = ['TRBC_ECONOMIC_SECTOR', 'TRBC_BUSINESS_SECTOR', 'TRBC_INDUSTRY_GROUP', 'TRBC_INDUSTRY', 'TRBC_ACTIVITY']

TRBC fields available (deprecated).

add(field: Union[str, list[str]], op: str, value: Union[int, float, str, tuple], frequency: Union[str, list[str]] = None, aggr_method=None, percent: bool = False, analyst_estimates=False, **kwargs) → None

Add a filter to the list of filters.

Parameters:

  • field – Fundamental field, history field, IBES analyst estimates field, list of fundamental/history fields.

  • op – Filter operator.

  • value – Value/threshold.

  • frequency – Field frequency or list of field frequencies (only required for fundamental and history fields).

  • aggr_method – Customised aggregation method (optional).

  • percent – Value parameter is in percentage (default is False).

  • analyst_estimates – Filter is based on IBES analyst estimates field (default is False).

  • kwargs – Additional keyword arguments, e.g. metric and historical_period_index for IBES analyst estimates.

apply_filter(dates: Union[Timestamp, date, datetime, list[datetime.date], list[datetime.datetime], DatetimeIndex], ignore_asofdate: bool = True) → Series

Filter a stock universe according to a set of filters.

Parameters:

  • dates – A date, datetime, pandas timestamp or set of datetimes (pandas DatetimeIndex). If the universe is represented by an index, the index constituents on the dates provided are used. If the universe is represented by a customised dict, the dates provided should be in the set of universe dates.

  • ignore_asofdate – If set to True, timeseries data are retrieved using the as_of_date of the environment. If set to False, the as_of_date of the timeseries coincides with the date in which the filter is applied (when applied to multiple dates, this setting may be more computationally expensive).

Returns:

A pandas series (date: list of filtered stock identifiers).

available_fields(stock_id: str) → dict

History fields and fundamental fields available for an underlying.

Parameters:

stock_id – Stock identifier.

Returns:

Dict of history and fundamental fields with list of available frequencies.

available_frequencies(field: str, date: Union[Timestamp, datetime] = None) → list[str]

Return a list of frequencies available for all stocks in the universe.

Parameters:

  • field – Fundamental or history field.

  • date – An optional point in time for the universe (timestamp or datetime). If not provided, the universe considered is the superset of all the individual points in time.

Returns:

List of frequencies available for all stocks in the universe.

clean() → None

Empty the list of filters defined by the user.

diff_additions() → Optional[Series]

Return the stocks added at every point in time given last timeseries computed by apply_filter. If less that two data points are available, the method returns None.

For instance, given the following filtered stocks:

` 2019-01-15: [Z, B] 2019-01-16: [Z, E] 2019-01-17: [Z, E] 2019-01-18: [Z, D] 2019-01-21: [Z, A] `

The method returns:

` 2019-01-16: [E] 2019-01-17: [] 2019-01-18: [D] 2019-01-21: [A] `

diff_deletions() → Optional[Series]

Return the stocks removed at every point in time given last timeseries computed by apply_filter. If less that two data points are available, the method returns None.

For instance, given the following filtered stocks:

` 2019-01-15: [Z, B] 2019-01-16: [Z, E] 2019-01-17: [Z, E] 2019-01-18: [Z, D] 2019-01-21: [Z, A] `

The method returns:

` 2019-01-16: [B] 2019-01-17: [] 2019-01-18: [E] 2019-01-21: [D] `

display_fields_data(date: Union[Timestamp, datetime], ids: list[str], fields: list[str], frequencies: list[str], ignore_asofdate: bool = True) → DataFrame

Return a DataFrame containing timeseries data defined by a date and a list of stock IDs, fields and frequencies.

Parameters:

  • date – A point in time (timestamp, date or datetime).

  • ids – List of stock identifiers.

  • fields – List of field identifiers.

  • frequencies – List of field frequencies.

  • ignore_asofdate – If set to True, timeseries data are retrieved using the as_of_date of the environment. If set to False, the as_of_date of the timeseries coincides with the display date (this setting may be more computationally expensive).

Returns:

pandas.DataFrame.

exclude_stocks(stock_ids: str | list[str])

Create a restriction list by excluding single stocks from the equity universe filter.

Parameters:

stock_ids – List of stock identifiers which will not be loaded and considered for the universe filtering.

list() → list

Return the list of filters defined by the user.

universe_ids(date: Union[Timestamp, date, datetime]) → list

Return a list of stock IDs representing the universe at a point in time. The output is affected by the flag most_liquid set in the constructor.

Parameters:

date – A point in time (timestamp, date or datetime).

Returns:

List of stock IDs representing the universe at a point in time.