Reworks how we choose a benchmark source for the zipline CLI and
zipline.utils.run_algo.run_algorithm. The logic for choosing a benchmark is now
consolidated in a new `BenchmarkSpec` class.
Benchmarks can be configured from several possible options:
- An in-memory series can be passed as `benchmark_returns`. This is mostly
useful for testing, or for use with `run_algorithm`.
- A path or file-like object can be passed as `benchmark_file`.
- A sid can be passed as `benchmark_sid`
- A symbol can be passed as `benchmark_symbol`.
- `no_benchmark` can be passed to use a dummy benchmark of all zero returns.
`BenchmarkSpec` takes all the parameters listed above, and "resolves" them into
either a sid or a series of benchmark returns, which are forwarded to
TradingAlgorithm, which already takes `benchmark_returns` or
`benchmark_sid`. If none of the above parameters are passed, we log a warning,
emit None for both values, and assume that the algorithm will call
`set_benchmark` in its `initialize` method.
MAINT: Remove environment as an argument to benchmark source.
To allow the BenchmarkSource class to be more easily used in contexts other than
a TradingAlgorithm, remove the TradingEnvironment as an argument to the
benchmark source.
Instead:
- Pass a benchmark Asset, instead of a bencmark sid; so that the asset_finder
does not need to be passed to the benchmark source.
- Pass the pre-calculated benchmark_returns instead of an env,
which contains the benchmark_returns; a consumer can let the benchmark_returns
stay as the default of `None` when using an asset.
We may want to further refactor and make two different classes, instead of
relying on a combination of existence/non-existence of benchmark_asset and
benchmark_returns. That refactoring should be easier to do with this change.
This allows us to remove the check for whether the provided dt had a
time of midnight, which was a flimsy way to infer if the data frequency
was 'daily'. Besides the explicit check being preferable, this method
was broken on the futures calendar, since midnight is a valid market
minute.
When adding fixtures for futures data, there will be a need for multiple
calendars in the fixture ecosystem. e.g. a test that includes both
equities and futures would need an overall calendar which encompasses
both equities and futures; however, the test data for equities should
still still be limited to the bounds set by the NYSE calendar.
Make the fixtures that setup trading calendars and values dervied from
the trading calendar (e.g. trading sessions) accept an iterable of
calendars which need to be created, then populate those values into a
dict keyed by the calendar name.
Change `WithNYSETradingDays` to include sessions in the name,
since we are moving to session as the name for the 'day' unit.
Provide `trading_days` which is really "NYSE trading sessions` on
`WithTradingSessions` for backwards compatibility.
Changes the overlap behavior so that it is an error to write data which
would have two companies holding the same ticker. Other than one test
around which company would win in that case, all the other tests are
passing. That single test has been changed to check the write-time
error.
Instead of having separate ExchangeCalendar and TradingSchedule objects, we
now just have TradingCalendar. The TradingCalendar keeps track of each
session (defined as a contiguous set of minutes between an open and a close).
It's also responsible for handling the grouping logic of any given minute
to its containing session, or the next/previous session if it's not a market
minute for the given calendar.
In preparation of adding futures, add equity to the names of both the
classes and methods for writing bcolz data. Futures data will use a
different minutes per day with a separate reader. This change will allow
both equity and futures fixtures to be side by side.
Also, break out the method which generates the dataframes and trading
days member into fixtures (`EquityMinuteBarData` and
`EquityDailyBarData`) on which the `*BarReader` fixture depends. This
fixture is separated out to enable reader/writers in different formats
to use the same data setup. (There is internal code which needs to write
minute and daily bar data in a database format.)
Instead of inferring it from the minute/daily writer, we now require the
first trading day to be passed explicitly, so the creator of the
DataPortal controls what is used as the first trading day.
Changes BcolzDailyBarWriter to not be an abc, data is passed as an
iterator of (sid, dataframe) pairs to the write method.
Changes the AssetsDBWriter to be a single class which accepts an engine
at construction time and has a `write` method for writing dataframes for
the various tables. We no longer support writing the various other data
types, callers should coerce their data into a dataframe themselves. See
zipline.assets.synthetic for some helpers to do this.
Adds many new fixtures and updates some existing fixtures to use the new
ones:
WithDefaultDateBounds
A fixture that provides the suite a START_DATE and END_DATE. This is
meant to make it easy for other fixtures to synchronize their date
ranges without depending on eachother in strange ways. For example,
WithBcolzMinuteBarReader and WithBcolzDailyBarReader by default should
both have data for the same dates, so they may use depend on
WithDefaultDates without forcing a dependency between them.
WithTmpDir, WithInstanceTmpDir
Provides the suite or individual test case a temporary directory.
WithBcolzDailyBarReader
Provides the suite a BcolzDailyBarReader which reads from bcolz data
written to a temporary directory. The data will be read from
dataframes and then converted to bcolz files with
BcolzDailyBarWriter.write
WithBcolzDailyBarReaderFromCSVs
Provides the suite a BcolzDailyBarReader which reads from bcolz data
written to a temporary directory. The data will be read from a
collection of CSV files and then converted into the bcolz data through
BcolzDailyBarWriter.write_csvs
WithBcolzMinuteBarReader
Provides the suite a BcolzMinuteBarReader which reads from bcolz data
written to a temporary directory. The data will be read from
dataframes and then converted to bcolz files with
BcolzMinuteBarWriter.write
WithAdjustmentReader
Provides the suite a SQLiteAdjustmentReader which reads from an in
memory sqlite database. The data will be read from dataframes and then
converted into sqlite with SQLiteAdjustmentWriter.write
WithDataPortal
Provides each test case a DataPortal object with data from temporary
resources.
More documentation to follow in release notes.
Based on lazy-mainline branch, see for more details.
Also-By: Jean Bredeche <jean@quantopian.com>
Also-By: Andrew Liang <aliang@quantopian.com>
Also-By: Abhijeet Kalyan <akalyan@quantopian.com>
