mirror of
https://github.com/saymrwulf/pytorch.git
synced 2026-05-14 20:57:59 +00:00
b18eeaa80a
Summary:
Implements `np.diff` for single order differences only:
- method and function variants for `diff` and function variant for `diff_out`
- supports out variant, but not in-place since shape changes
- adds OpInfo entry, and test in `test_torch`
- automatic autograd because we are using the `Math` dispatch
_Update: we only support Tensors for prepend and append in this PR. See discussion below and comments for more details._
Currently there is a quirk in the c++ API based on how this is implemented: it is not possible to specify scalar prepend and appends without also specifying all 4 arguments.
That is because the goal is to match NumPy's diff signature of `diff(int n=1, int dim=-1, Union[Scalar, Tensor] prepend=None, Union[Scalar, Tensor] append)=None` where all arguments are optional, positional and in the correct order.
There are a couple blockers. One is c++ ambiguity. This prevents us from simply doing `diff(int n=1, int dim=-1, Scalar? prepend=None, Tensor? append=None)` etc for all combinations of {Tensor, Scalar} x {Tensor, Scalar}.
Why not have append, prepend not have default args and then write out the whole power set of {Tensor, Scalar, omitted} x {Tensor, Scalar, omitted} you might ask. Aside from having to write 18 overloads, this is actually illegal because arguments with defaults must come after arguments without defaults. This would mean having to write `diff(prepend, append, n, dim)` which is not desired. Finally writing out the entire power set of all arguments n, dim, prepend, append is out of the question because that would actually involve 2 * 2 * 3 * 3 = 36 combinations. And if we include the out variant, that would be 72 overloads!
With this in mind, the current way this is implemented is actually to still do `diff(int n=1, int dim=-1, Scalar? prepend=None, Tensor? append=None)`. But also make use of `cpp_no_default_args`. The idea is to only have one of the 4 {Tensor, Scalar} x {Tensor, Scalar} provide default arguments for the c++ api, and add `cpp_no_default_args` for the remaining 3 overloads. With this, Python api works as expected, but some calls such as `diff(prepend=1)` won't work on c++ api.
We can optionally add 18 more overloads that cover the {dim, n, no-args} x {scalar-tensor, tensor-scalar, scalar-scalar} x {out, non-out} cases for c++ api. _[edit: counting is hard - just realized this number is still wrong. We should try to count the cases we do cover instead and subtract that from the total: (2 * 2 * 3 * 3) - (3 + 2^4) = 17. 3 comes from the 3 of 4 combinations of {tensor, scalar}^2 that we declare to be `cpp_no_default_args`, and the one remaining case that has default arguments has covers 2^4 cases. So actual count is 34 additional overloads to support all possible calls]_
_[edit: thanks to https://github.com/pytorch/pytorch/issues/50767 hacky_wrapper is no longer necessary; it is removed in the latest commit]_
hacky_wrapper was also necessary here because `Tensor?` will cause dispatch to look for the `const optional<Tensor>&` schema but also generate a `const Tensor&` declaration in Functions.h. hacky_wrapper allows us to define our function as `const Tensor&` but wraps it in optional for us, so this avoids both the errors while linking and loading.
_[edit: rewrote the above to improve clarity and correct the fact that we actually need 18 more overloads (26 total), not 18 in total to complete the c++ api]_
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50569
Reviewed By: H-Huang
Differential Revision: D26176105
Pulled By: soulitzer
fbshipit-source-id: cd8e77cc2de1117c876cd71c29b312887daca33f
565 lines
9.2 KiB
ReStructuredText
565 lines
9.2 KiB
ReStructuredText
torch
|
|
=====
|
|
The torch package contains data structures for multi-dimensional
|
|
tensors and defines mathematical operations over these tensors.
|
|
Additionally, it provides many utilities for efficient serializing of
|
|
Tensors and arbitrary types, and other useful utilities.
|
|
|
|
It has a CUDA counterpart, that enables you to run your tensor computations
|
|
on an NVIDIA GPU with compute capability >= 3.0
|
|
|
|
.. currentmodule:: torch
|
|
|
|
Tensors
|
|
-------
|
|
.. autosummary::
|
|
:toctree: generated
|
|
:nosignatures:
|
|
|
|
is_tensor
|
|
is_storage
|
|
is_complex
|
|
is_floating_point
|
|
is_nonzero
|
|
set_default_dtype
|
|
get_default_dtype
|
|
set_default_tensor_type
|
|
numel
|
|
set_printoptions
|
|
set_flush_denormal
|
|
|
|
.. _tensor-creation-ops:
|
|
|
|
Creation Ops
|
|
~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. note::
|
|
Random sampling creation ops are listed under :ref:`random-sampling` and
|
|
include:
|
|
:func:`torch.rand`
|
|
:func:`torch.rand_like`
|
|
:func:`torch.randn`
|
|
:func:`torch.randn_like`
|
|
:func:`torch.randint`
|
|
:func:`torch.randint_like`
|
|
:func:`torch.randperm`
|
|
You may also use :func:`torch.empty` with the :ref:`inplace-random-sampling`
|
|
methods to create :class:`torch.Tensor` s with values sampled from a broader
|
|
range of distributions.
|
|
|
|
.. autosummary::
|
|
:toctree: generated
|
|
:nosignatures:
|
|
|
|
tensor
|
|
sparse_coo_tensor
|
|
as_tensor
|
|
as_strided
|
|
from_numpy
|
|
zeros
|
|
zeros_like
|
|
ones
|
|
ones_like
|
|
arange
|
|
range
|
|
linspace
|
|
logspace
|
|
eye
|
|
empty
|
|
empty_like
|
|
empty_strided
|
|
full
|
|
full_like
|
|
quantize_per_tensor
|
|
quantize_per_channel
|
|
dequantize
|
|
complex
|
|
polar
|
|
heaviside
|
|
|
|
Indexing, Slicing, Joining, Mutating Ops
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
.. autosummary::
|
|
:toctree: generated
|
|
:nosignatures:
|
|
|
|
cat
|
|
chunk
|
|
column_stack
|
|
dstack
|
|
gather
|
|
hstack
|
|
index_select
|
|
masked_select
|
|
movedim
|
|
moveaxis
|
|
narrow
|
|
nonzero
|
|
reshape
|
|
row_stack
|
|
scatter
|
|
scatter_add
|
|
split
|
|
squeeze
|
|
stack
|
|
swapaxes
|
|
swapdims
|
|
t
|
|
take
|
|
tensor_split
|
|
tile
|
|
transpose
|
|
unbind
|
|
unsqueeze
|
|
vstack
|
|
where
|
|
|
|
.. _generators:
|
|
|
|
Generators
|
|
----------------------------------
|
|
.. autosummary::
|
|
:toctree: generated
|
|
:nosignatures:
|
|
|
|
Generator
|
|
|
|
.. _random-sampling:
|
|
|
|
Random sampling
|
|
----------------------------------
|
|
.. autosummary::
|
|
:toctree: generated
|
|
:nosignatures:
|
|
|
|
seed
|
|
manual_seed
|
|
initial_seed
|
|
get_rng_state
|
|
set_rng_state
|
|
|
|
.. autoattribute:: torch.default_generator
|
|
:annotation: Returns the default CPU torch.Generator
|
|
|
|
.. The following doesn't actually seem to exist.
|
|
https://github.com/pytorch/pytorch/issues/27780
|
|
.. autoattribute:: torch.cuda.default_generators
|
|
:annotation: If cuda is available, returns a tuple of default CUDA torch.Generator-s.
|
|
The number of CUDA torch.Generator-s returned is equal to the number of
|
|
GPUs available in the system.
|
|
.. autosummary::
|
|
:toctree: generated
|
|
:nosignatures:
|
|
|
|
bernoulli
|
|
multinomial
|
|
normal
|
|
poisson
|
|
rand
|
|
rand_like
|
|
randint
|
|
randint_like
|
|
randn
|
|
randn_like
|
|
randperm
|
|
|
|
.. _inplace-random-sampling:
|
|
|
|
In-place random sampling
|
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
There are a few more in-place random sampling functions defined on Tensors as well. Click through to refer to their documentation:
|
|
|
|
- :func:`torch.Tensor.bernoulli_` - in-place version of :func:`torch.bernoulli`
|
|
- :func:`torch.Tensor.cauchy_` - numbers drawn from the Cauchy distribution
|
|
- :func:`torch.Tensor.exponential_` - numbers drawn from the exponential distribution
|
|
- :func:`torch.Tensor.geometric_` - elements drawn from the geometric distribution
|
|
- :func:`torch.Tensor.log_normal_` - samples from the log-normal distribution
|
|
- :func:`torch.Tensor.normal_` - in-place version of :func:`torch.normal`
|
|
- :func:`torch.Tensor.random_` - numbers sampled from the discrete uniform distribution
|
|
- :func:`torch.Tensor.uniform_` - numbers sampled from the continuous uniform distribution
|
|
|
|
Quasi-random sampling
|
|
~~~~~~~~~~~~~~~~~~~~~
|
|
.. autosummary::
|
|
:toctree: generated
|
|
:nosignatures:
|
|
:template: sobolengine.rst
|
|
|
|
quasirandom.SobolEngine
|
|
|
|
Serialization
|
|
----------------------------------
|
|
.. autosummary::
|
|
:toctree: generated
|
|
:nosignatures:
|
|
|
|
save
|
|
load
|
|
|
|
Parallelism
|
|
----------------------------------
|
|
.. autosummary::
|
|
:toctree: generated
|
|
:nosignatures:
|
|
|
|
get_num_threads
|
|
set_num_threads
|
|
get_num_interop_threads
|
|
set_num_interop_threads
|
|
|
|
Locally disabling gradient computation
|
|
--------------------------------------
|
|
The context managers :func:`torch.no_grad`, :func:`torch.enable_grad`, and
|
|
:func:`torch.set_grad_enabled` are helpful for locally disabling and enabling
|
|
gradient computation. See :ref:`locally-disable-grad` for more details on
|
|
their usage. These context managers are thread local, so they won't
|
|
work if you send work to another thread using the ``threading`` module, etc.
|
|
|
|
Examples::
|
|
|
|
>>> x = torch.zeros(1, requires_grad=True)
|
|
>>> with torch.no_grad():
|
|
... y = x * 2
|
|
>>> y.requires_grad
|
|
False
|
|
|
|
>>> is_train = False
|
|
>>> with torch.set_grad_enabled(is_train):
|
|
... y = x * 2
|
|
>>> y.requires_grad
|
|
False
|
|
|
|
>>> torch.set_grad_enabled(True) # this can also be used as a function
|
|
>>> y = x * 2
|
|
>>> y.requires_grad
|
|
True
|
|
|
|
>>> torch.set_grad_enabled(False)
|
|
>>> y = x * 2
|
|
>>> y.requires_grad
|
|
False
|
|
|
|
.. autosummary::
|
|
:toctree: generated
|
|
:nosignatures:
|
|
|
|
no_grad
|
|
enable_grad
|
|
set_grad_enabled
|
|
|
|
Math operations
|
|
---------------
|
|
|
|
Pointwise Ops
|
|
~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autosummary::
|
|
:toctree: generated
|
|
:nosignatures:
|
|
|
|
abs
|
|
absolute
|
|
acos
|
|
arccos
|
|
acosh
|
|
arccosh
|
|
add
|
|
addcdiv
|
|
addcmul
|
|
angle
|
|
asin
|
|
arcsin
|
|
asinh
|
|
arcsinh
|
|
atan
|
|
arctan
|
|
atanh
|
|
arctanh
|
|
atan2
|
|
bitwise_not
|
|
bitwise_and
|
|
bitwise_or
|
|
bitwise_xor
|
|
ceil
|
|
clamp
|
|
clip
|
|
conj
|
|
copysign
|
|
cos
|
|
cosh
|
|
deg2rad
|
|
div
|
|
divide
|
|
digamma
|
|
erf
|
|
erfc
|
|
erfinv
|
|
exp
|
|
exp2
|
|
expm1
|
|
fix
|
|
float_power
|
|
floor
|
|
floor_divide
|
|
fmod
|
|
frac
|
|
imag
|
|
ldexp
|
|
lerp
|
|
lgamma
|
|
log
|
|
log10
|
|
log1p
|
|
log2
|
|
logaddexp
|
|
logaddexp2
|
|
logical_and
|
|
logical_not
|
|
logical_or
|
|
logical_xor
|
|
logit
|
|
hypot
|
|
i0
|
|
igamma
|
|
igammac
|
|
mul
|
|
multiply
|
|
mvlgamma
|
|
nan_to_num
|
|
neg
|
|
negative
|
|
nextafter
|
|
polygamma
|
|
pow
|
|
rad2deg
|
|
real
|
|
reciprocal
|
|
remainder
|
|
round
|
|
rsqrt
|
|
sigmoid
|
|
sign
|
|
sgn
|
|
signbit
|
|
sin
|
|
sinc
|
|
sinh
|
|
sqrt
|
|
square
|
|
sub
|
|
subtract
|
|
tan
|
|
tanh
|
|
true_divide
|
|
trunc
|
|
xlogy
|
|
|
|
Reduction Ops
|
|
~~~~~~~~~~~~~~~~~~~~~~
|
|
.. autosummary::
|
|
:toctree: generated
|
|
:nosignatures:
|
|
|
|
argmax
|
|
argmin
|
|
amax
|
|
amin
|
|
all
|
|
any
|
|
max
|
|
min
|
|
dist
|
|
logsumexp
|
|
mean
|
|
median
|
|
nanmedian
|
|
mode
|
|
norm
|
|
nansum
|
|
prod
|
|
quantile
|
|
nanquantile
|
|
std
|
|
std_mean
|
|
sum
|
|
unique
|
|
unique_consecutive
|
|
var
|
|
var_mean
|
|
count_nonzero
|
|
|
|
Comparison Ops
|
|
~~~~~~~~~~~~~~~~~~~~~~
|
|
.. autosummary::
|
|
:toctree: generated
|
|
:nosignatures:
|
|
|
|
allclose
|
|
argsort
|
|
eq
|
|
equal
|
|
ge
|
|
greater_equal
|
|
gt
|
|
greater
|
|
isclose
|
|
isfinite
|
|
isinf
|
|
isposinf
|
|
isneginf
|
|
isnan
|
|
isreal
|
|
kthvalue
|
|
le
|
|
less_equal
|
|
lt
|
|
less
|
|
maximum
|
|
minimum
|
|
fmax
|
|
fmin
|
|
ne
|
|
not_equal
|
|
sort
|
|
topk
|
|
msort
|
|
|
|
|
|
Spectral Ops
|
|
~~~~~~~~~~~~~~~~~~~~~~
|
|
.. autosummary::
|
|
:toctree: generated
|
|
:nosignatures:
|
|
|
|
stft
|
|
istft
|
|
bartlett_window
|
|
blackman_window
|
|
hamming_window
|
|
hann_window
|
|
kaiser_window
|
|
|
|
|
|
Other Operations
|
|
~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autosummary::
|
|
:toctree: generated
|
|
:nosignatures:
|
|
|
|
atleast_1d
|
|
atleast_2d
|
|
atleast_3d
|
|
bincount
|
|
block_diag
|
|
broadcast_tensors
|
|
broadcast_to
|
|
broadcast_shapes
|
|
bucketize
|
|
cartesian_prod
|
|
cdist
|
|
clone
|
|
combinations
|
|
cross
|
|
cummax
|
|
cummin
|
|
cumprod
|
|
cumsum
|
|
diag
|
|
diag_embed
|
|
diagflat
|
|
diagonal
|
|
diff
|
|
einsum
|
|
flatten
|
|
flip
|
|
fliplr
|
|
flipud
|
|
kron
|
|
rot90
|
|
gcd
|
|
histc
|
|
meshgrid
|
|
lcm
|
|
logcumsumexp
|
|
ravel
|
|
renorm
|
|
repeat_interleave
|
|
roll
|
|
searchsorted
|
|
tensordot
|
|
trace
|
|
tril
|
|
tril_indices
|
|
triu
|
|
triu_indices
|
|
vander
|
|
view_as_real
|
|
view_as_complex
|
|
|
|
|
|
BLAS and LAPACK Operations
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
.. autosummary::
|
|
:toctree: generated
|
|
:nosignatures:
|
|
|
|
addbmm
|
|
addmm
|
|
addmv
|
|
addr
|
|
baddbmm
|
|
bmm
|
|
chain_matmul
|
|
cholesky
|
|
cholesky_inverse
|
|
cholesky_solve
|
|
dot
|
|
eig
|
|
geqrf
|
|
ger
|
|
inner
|
|
inverse
|
|
det
|
|
logdet
|
|
slogdet
|
|
lstsq
|
|
lu
|
|
lu_solve
|
|
lu_unpack
|
|
matmul
|
|
matrix_power
|
|
matrix_rank
|
|
matrix_exp
|
|
mm
|
|
mv
|
|
orgqr
|
|
ormqr
|
|
outer
|
|
pinverse
|
|
qr
|
|
solve
|
|
svd
|
|
svd_lowrank
|
|
pca_lowrank
|
|
symeig
|
|
lobpcg
|
|
trapz
|
|
triangular_solve
|
|
vdot
|
|
|
|
Utilities
|
|
----------------------------------
|
|
.. autosummary::
|
|
:toctree: generated
|
|
:nosignatures:
|
|
|
|
compiled_with_cxx11_abi
|
|
result_type
|
|
can_cast
|
|
promote_types
|
|
use_deterministic_algorithms
|
|
are_deterministic_algorithms_enabled
|
|
vmap
|
|
_assert
|