saymrwulf/pytorch
1
0
Fork 0
mirror of https://github.com/saymrwulf/pytorch.git synced 2026-05-14 20:57:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
pytorch/caffe2
History
Kaustubh Vartak 78f30386c5 Implement Swish(SiLU) operator in FP16 
Summary:
Used Caffe2 Swish implemenmtation to implement the operator. Will need
to resolve the error introduced.
```
test_quantized_swish_2D (tests.operators.testQuantizedSilu.TestSiLU) ... input:
 (tensor([[-6.0000, -5.9961, -5.9922,  ..., -5.7734, -5.7695, -5.7656],
        [-5.7617, -5.7539, -5.7500,  ..., -5.5352, -5.5312, -5.5234],
        [-5.5195, -5.5156, -5.5117,  ..., -5.2930, -5.2891, -5.2852],
        ...,
        [ 5.2852,  5.2891,  5.2930,  ...,  5.5117,  5.5156,  5.5195],
        [ 5.5234,  5.5312,  5.5352,  ...,  5.7500,  5.7539,  5.7617],
        [ 5.7656,  5.7695,  5.7734,  ...,  5.9922,  5.9961,  6.0000]]),)
base_res:
 tensor([[-0.0148, -0.0149, -0.0149,  ..., -0.0179, -0.0180, -0.0180],
        [-0.0181, -0.0182, -0.0182,  ..., -0.0218, -0.0218, -0.0220],
        [-0.0220, -0.0221, -0.0222,  ..., -0.0265, -0.0266, -0.0266],
        ...,
        [ 5.2585,  5.2625,  5.2665,  ...,  5.4895,  5.4935,  5.4975],
        [ 5.5015,  5.5094,  5.5134,  ...,  5.7318,  5.7357,  5.7437],
        [ 5.7476,  5.7516,  5.7555,  ...,  5.9773,  5.9812,  5.9852]])
tnco_res:
 tensor([[-0.0148, -0.0149, -0.0149,  ..., -0.0179, -0.0180, -0.0180],
        [-0.0181, -0.0182, -0.0182,  ..., -0.0218, -0.0218, -0.0220],
        [-0.0220, -0.0221, -0.0222,  ..., -0.0265, -0.0265, -0.0266],
        ...,
        [ 5.2578,  5.2617,  5.2656,  ...,  5.4922,  5.4922,  5.4961],
        [ 5.5000,  5.5078,  5.5156,  ...,  5.7305,  5.7383,  5.7422],
        [ 5.7461,  5.7500,  5.7539,  ...,  5.9766,  5.9805,  5.9844]])
nnpi_res:
 tensor([[-0.0148, -0.0149, -0.0149,  ..., -0.0179, -0.0180, -0.0180],
        [-0.0181, -0.0182, -0.0182,  ..., -0.0218, -0.0218, -0.0220],
        [-0.0220, -0.0221, -0.0222,  ..., -0.0265, -0.0266, -0.0266],
        ...,
        [ 5.2585,  5.2625,  5.2665,  ...,  5.4895,  5.4935,  5.4975],
        [ 5.5015,  5.5094,  5.5134,  ...,  5.7318,  5.7357,  5.7437],
        [ 5.7476,  5.7516,  5.7555,  ...,  5.9773,  5.9812,  5.9852]])
diff:
 tensor([[4.1956e-06, 9.8441e-07, 6.0154e-06,  ..., 4.2785e-06, 7.6480e-06,
         1.0842e-05],
        [1.3988e-06, 4.1034e-06, 6.5863e-06,  ..., 5.3961e-06, 2.9635e-06,
         1.0209e-05],
        [1.2219e-06, 7.9758e-06, 1.7386e-05,  ..., 3.0547e-07, 2.2141e-05,
         1.4316e-05],
        ...,
        [7.0286e-04, 7.8678e-04, 8.7023e-04,  ..., 2.6422e-03, 1.3347e-03,
         1.4052e-03],
        [1.4753e-03, 1.6141e-03, 2.2225e-03,  ..., 1.2884e-03, 2.5592e-03,
         1.4634e-03],
        [1.5216e-03, 1.5793e-03, 1.6365e-03,  ..., 6.9284e-04, 7.4100e-04,
         7.8964e-04]])
nnpi traced graph:
 graph(%self : __torch__.tests.operators.testQuantizedSilu.SiLUModel,
      %x : Float(*, *, requires_grad=0, device=cpu)):
  %3 : None = prim::Constant()
  %4 : bool = prim::Constant[value=0]() # /data/users/kaus/fbsource/fbcode/buck-out/dev/gen/glow/fb/torch_glow/custom_nnpi_ops/testQuantizedSilu#binary,link-tree/tests/operators/testQuantizedSilu.py:13:0
  %5 : Device = prim::Constant[value="cpu"]() # /data/users/kaus/fbsource/fbcode/buck-out/dev/gen/glow/fb/torch_glow/custom_nnpi_ops/testQuantizedSilu#binary,link-tree/tests/operators/testQuantizedSilu.py:13:0
  %6 : int = prim::Constant[value=0]() # /data/users/kaus/fbsource/fbcode/buck-out/dev/gen/glow/fb/torch_glow/custom_nnpi_ops/testQuantizedSilu#binary,link-tree/tests/operators/testQuantizedSilu.py:13:0
  %7 : int = prim::Constant[value=6]() # /data/users/kaus/fbsource/fbcode/buck-out/dev/gen/glow/fb/torch_glow/custom_nnpi_ops/testQuantizedSilu#binary,link-tree/tests/operators/testQuantizedSilu.py:13:0
  %8 : Float(*, *, requires_grad=0, device=cpu) = aten::zeros_like(%x, %7, %6, %5, %4, %3) # /data/users/kaus/fbsource/fbcode/buck-out/dev/gen/glow/fb/torch_glow/custom_nnpi_ops/testQuantizedSilu#binary,link-tree/tests/operators/testQuantizedSilu.py:13:0
  %input : Float(*, *, requires_grad=0, device=cpu) = glow::FusionGroup_0(%x, %8)
  %10 : Tensor = aten::silu(%input) # /data/users/kaus/fbsource/fbcode/buck-out/dev/gen/glow/fb/torch_glow/custom_nnpi_ops/testQuantizedSilu#binary,link-tree/torch/nn/functional.py:1804:0
  return (%10)
with glow::FusionGroup_0 = graph(%0 : Float(*, *, requires_grad=0, device=cpu),
      %1 : Float(*, *, requires_grad=0, device=cpu)):
  %2 : int = prim::Constant[value=1]()
  %input : Float(*, *, requires_grad=0, device=cpu) = aten::add(%0, %1, %2) # /data/users/kaus/fbsource/fbcode/buck-out/dev/gen/glow/fb/torch_glow/custom_nnpi_ops/testQuantizedSilu#binary,link-tree/tests/operators/testQuantizedSilu.py:13:0
  %4 : int = prim::Constant[value=1]() # /data/users/kaus/fbsource/fbcode/buck-out/dev/gen/glow/fb/torch_glow/custom_nnpi_ops/testQuantizedSilu#binary,link-tree/tests/operators/testQuantizedSilu.py:13:0
  return (%input)

tnco traced graph:
 graph(%self : __torch__.tests.operators.testQuantizedSilu.___torch_mangle_0.SiLUModel,
      %x : Float(*, *, requires_grad=0, device=cpu)):
  %2 : int = prim::Constant[value=1]() # /data/users/kaus/fbsource/fbcode/buck-out/dev/gen/glow/fb/torch_glow/custom_nnpi_ops/testQuantizedSilu#binary,link-tree/tests/operators/testQuantizedSilu.py:13:0
  %3 : None = prim::Constant()
  %4 : bool = prim::Constant[value=0]() # /data/users/kaus/fbsource/fbcode/buck-out/dev/gen/glow/fb/torch_glow/custom_nnpi_ops/testQuantizedSilu#binary,link-tree/tests/operators/testQuantizedSilu.py:13:0
  %5 : Device = prim::Constant[value="cpu"]() # /data/users/kaus/fbsource/fbcode/buck-out/dev/gen/glow/fb/torch_glow/custom_nnpi_ops/testQuantizedSilu#binary,link-tree/tests/operators/testQuantizedSilu.py:13:0
  %6 : int = prim::Constant[value=0]() # /data/users/kaus/fbsource/fbcode/buck-out/dev/gen/glow/fb/torch_glow/custom_nnpi_ops/testQuantizedSilu#binary,link-tree/tests/operators/testQuantizedSilu.py:13:0
  %7 : int = prim::Constant[value=6]() # /data/users/kaus/fbsource/fbcode/buck-out/dev/gen/glow/fb/torch_glow/custom_nnpi_ops/testQuantizedSilu#binary,link-tree/tests/operators/testQuantizedSilu.py:13:0
  %8 : Float(*, *, requires_grad=0, device=cpu) = aten::zeros_like(%x, %7, %6, %5, %4, %3) # /data/users/kaus/fbsource/fbcode/buck-out/dev/gen/glow/fb/torch_glow/custom_nnpi_ops/testQuantizedSilu#binary,link-tree/tests/operators/testQuantizedSilu.py:13:0
  %12 : Tensor = fakeNNPI::addFP16(%x, %8, %2)
  %11 : Tensor = fakeNNPI::siluFP16(%12)
  return (%11)

FAIL

======================================================================
FAIL: test_quantized_swish_2D (tests.operators.testQuantizedSilu.TestSiLU)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/users/kaus/fbsource/fbcode/buck-out/dev/gen/glow/fb/torch_glow/custom_nnpi_ops/testQuantizedSilu#binary,link-tree/tests/operators/testQuantizedSilu.py", line 26, in test_quantized_swish_2D
    validate_nnpi_model(model, (x,), expected_ops, [])
  File "/data/users/kaus/fbsource/fbcode/buck-out/dev/gen/glow/fb/torch_glow/custom_nnpi_ops/testQuantizedSilu#binary,link-tree/tests/utils.py", line 73, in validate_nnpi_model
    assert is_equal
AssertionError
```

Test Plan:
Run test with buck test  mode/dev
//glow/fb/torch_glow/custom_nnpi_ops:testQuantizedSilu

Reviewed By: hyuen

Differential Revision: D25981369

fbshipit-source-id: dd0f3686b3cbf6fc575c959c7661125ecbf0b0db
2021-01-22 13:57:54 -08:00
..
contrib Implement Swish(SiLU) operator in FP16 2021-01-22 13:57:54 -08:00
core Introducing TORCH_CUDA_CPP_API and TORCH_CUDA_CU_API to the code (#50627) 2021-01-21 19:09:11 -08:00
cuda_rtc
db
distributed Renaming CAFFE2_API to TORCH_API (#49496) 2020-12-18 10:54:50 -08:00
experiments Remove __future__ imports for legacy Python2 supports (#45033) 2020-09-23 17:57:02 -07:00
ideep Upgrade oneDNN (mkl-dnn) to v1.7 (#47853) 2020-12-03 11:54:31 -08:00
image
mobile [PyTorch Mobile] Add continuous build config for xplat/caffe2 2020-12-09 16:58:20 -08:00
mpi Renaming CAFFE2_API to TORCH_API (#49496) 2020-12-18 10:54:50 -08:00
observers Renaming CAFFE2_API to TORCH_API (#49496) 2020-12-18 10:54:50 -08:00
onnx Renaming CAFFE2_API to TORCH_API (#49496) 2020-12-18 10:54:50 -08:00
operators [PyTorch] Remove CAFFE2_FB_LIMITED_MOBILE_CAPABILITY (#50385) 2021-01-20 10:26:54 -08:00
opt Remove instance of blacklist (#50478) 2021-01-19 14:42:01 -08:00
perfkernels Support int32 indices and offsets in nn.EmbeddingBag (#46758) 2020-11-03 23:33:50 -08:00
predictor Renaming CAFFE2_API to TORCH_API (#49496) 2020-12-18 10:54:50 -08:00
proto Renaming CAFFE2_API to TORCH_API (#49496) 2020-12-18 10:54:50 -08:00
python [BE] Fix the broken test caffe2/caffe2/python:lazy_dyndep_test - test_allcompare (#50696) 2021-01-18 16:21:06 -08:00
quantization Drop unused imports from caffe2/quantization (#50493) 2021-01-14 09:15:19 -08:00
queue Renaming CAFFE2_API to TORCH_API (#49496) 2020-12-18 10:54:50 -08:00
serialize [PyTorch] Decouple version numbers from c10 and caffe2 targets (#49905) 2020-12-30 15:34:01 -08:00
sgd [pytorch] [PR] Rename cuda kernel checks to C10 (#48615) 2020-12-01 18:19:07 -08:00
share [PyTorch] Remove CAFFE2_FB_LIMITED_MOBILE_CAPABILITY (#50385) 2021-01-20 10:26:54 -08:00
test
transforms Renaming CAFFE2_API to TORCH_API (#49496) 2020-12-18 10:54:50 -08:00
utils add and adjust kernel launch checks under fbcode/caffe2/caffe2/utils (#50862) 2021-01-21 15:20:55 -08:00
video Renaming CAFFE2_API to TORCH_API (#49496) 2020-12-18 10:54:50 -08:00
.clang-format
__init__.py
c2_aten_srcs.bzl
CMakeLists.txt Refactor build targets for torch::deploy (#50288) 2021-01-22 09:16:32 -08:00
README.md Grammar patch 1 (.md) (#41599) 2020-07-20 10:19:08 -07:00
release-notes.md
requirements.txt Remove references to typing module from setup.py (#47677) 2020-11-12 10:04:38 -08:00
VERSION_NUMBER

README.md

Caffe2

Jenkins Build Status

Caffe2 is a lightweight, modular, and scalable deep learning framework. Building on the original Caffe, Caffe2 is designed with expression, speed, and modularity in mind.

Questions and Feedback

Please use GitHub issues (https://github.com/pytorch/pytorch/issues) to ask questions, report bugs, and request new features.

Further Resources on Caffe2.ai