mirror of https://github.com/saymrwulf/pytorch.git synced 2026-05-15 21:00:47 +00:00

History

Igor Gitman 1b6d18aa7c Adding support for CuDNN-based LSTM with projections (#47725 ) Summary: Fixes https://github.com/pytorch/pytorch/issues/46213 I didn't yet update the documentation, will add those change soon. A few other things that I didn't do, but want to clarify if I maybe should. 1. I didn't expose projections in c++ API: torch/csrc/api/src/nn/modules/rnn.cpp. Let me know if this is desirable and I will add those changes. 2. I didn't expose projections in "lstm_cell" function and "_thnn_differentiable_lstm_cell_backward" functions from aten/src/ATen/native/RNN.cpp. As far as I understand, they are not needed for nn.LSTM CPU execution. For lstm_cell, projections don't bring any real benefit, since if cell is used separately, it can be easily added in Python. For "_thnn_differentiable_lstm_cell_backward", I'm actually not sure where exactly that function is used, so I also disabled projections there for now. Please let me know if I should change that. 3. I added check that projections are not supported for quantized LSTMs to quantized_lstm_<data/input> functions. But I didn't add any checks to LSTMCell code. It seems that since I disabled projections in "lstm_cell" function, they should also not be available for quantized models through any other API than quantized_lstm_<data/input>. Please let me know if I'm not correct and I will add checks to other places. 4. Projections are not supported for CuDNN versions < 7.1.2. Should I add the check for CuDNN version and disable projections in that case? If so, what will be the best way to do that? 5. Currently I added projection weight as the last weight, so the layout is "w_ih, w_hh, b_ih, b_hh, w_hr". This breaks the assumption that biases come after weights and thus I had to add additional if-s in various places. Alternative way would be to have "w_ih, w_hh, w_hr, b_ih, b_hh" layout, in which case the assumption will be true. But in that case I will need to split the loop in get_parameters function from aten/src/ATen/native/cudnn/RNN.cpp. And in some cases, I will still need to add an "undefined" tensor in the 3rd position, because we get all 5 weights from CuDNN most of the time. So I'm not sure which way is better. Let me know if you think I should change to the weights-then-biases layout. Pull Request resolved: https://github.com/pytorch/pytorch/pull/47725 Reviewed By: zou3519 Differential Revision: D25449794 Pulled By: ngimel fbshipit-source-id: fe6ce59e481d1f5fd861a8ff7fa13d1affcedb0c		2020-12-16 11:27:02 -08:00
..
any.cpp
autograd.cpp	Revert D24698027: Fix auto exponent issue for torch.pow	2020-11-15 03:58:44 -08:00
CMakeLists.txt	Implement C++ ModuleDict (#47707 )	2020-11-19 08:07:51 -08:00
dataloader.cpp
dispatch.cpp
enum.cpp
expanding-array.cpp
fft.cpp	Remove deprecated spectral ops from torch namespace (#48594 )	2020-12-05 04:12:32 -08:00
functional.cpp	[c++] Distance-agnostic triplet margin loss (#45377 )	2020-09-30 12:37:35 -07:00
init.cpp
init_baseline.h
init_baseline.py
integration.cpp
jit.cpp
memory.cpp
misc.cpp
module.cpp
moduledict.cpp	Implement C++ ModuleDict (#47707 )	2020-11-19 08:07:51 -08:00
modulelist.cpp
modules.cpp	Fix return-type-is-always-copy warning (#47279 )	2020-11-03 08:53:24 -08:00
namespace.cpp
nn_utils.cpp
operations.cpp	[Codemod][GleanFbcode] Remove dead includes in caffe2/test (#43953 )	2020-09-01 21:48:28 -07:00
optim.cpp
optim_baseline.h
optim_baseline.py
ordered_dict.cpp
parallel.cpp
parallel_benchmark.cpp
parameterdict.cpp
parameterlist.cpp
README.md
rnn.cpp	Adding support for CuDNN-based LSTM with projections (#47725 )	2020-12-16 11:27:02 -08:00
sequential.cpp
serialize.cpp
static.cpp
support.cpp
support.h
tensor.cpp
tensor_cuda.cpp
tensor_indexing.cpp
tensor_options.cpp	[PyTorch] Narrow Device to 2 bytes by narrowing DeviceType and DeviceIndex (#47023 )	2020-11-18 19:39:40 -08:00
tensor_options_cuda.cpp
torch_include.cpp
transformer.cpp	C++ APIs Transformer NN Module Top Layer (#44333 )	2020-09-11 08:25:27 -07:00

README.md

C++ Frontend Tests

In this folder live the tests for PyTorch's C++ Frontend. They use the GoogleTest test framework.

CUDA Tests

To make a test runnable only on platforms with CUDA, you should suffix your test with _CUDA, e.g.

TEST(MyTestSuite, MyTestCase_CUDA) { }

To make it runnable only on platforms with at least two CUDA machines, suffix it with _MultiCUDA instead of _CUDA, e.g.

TEST(MyTestSuite, MyTestCase_MultiCUDA) { }

There is logic in main.cpp that detects the availability and number of CUDA devices and supplies the appropriate negative filters to GoogleTest.

Integration Tests

Integration tests use the MNIST dataset. You must download it by running the following command from the PyTorch root folder:

$ python tools/download_mnist.py -d test/cpp/api/mnist

The required paths will be referenced as test/cpp/api/mnist/... in the test code, so you must run the integration tests from the PyTorch root folder.