saymrwulf/pytorch
1
0
Fork 0
mirror of https://github.com/saymrwulf/pytorch.git synced 2026-05-14 20:57:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
pytorch/test
History
Jason Carreiro fa62308673 [tensorboard] Fix TensorBoard summary encoding for torch.bfloat16 tensors (#108351) 
Summary:
The `tensor_proto` function in the TensorBoard summary writer code doesn't correctly encode `torch.bfloat16` tensors; it tries to use a data type of `DT_BFLOAT` when creating the protobuf, but `DT_BFLOAT` is not a valid enum value (see `types.proto`). The correct value to use when encoding tensors of this type is `DT_BLOAT16`. This diff updates the type map in the summary code to use the correct type.

While fixing this error, I also noticed the wrong field of the protobuf was being used when encoding tensors of this type; per the docs in the proto file, the DT_HALF and DT_BFLOAT16 types should use the `half_val` field, not `float_val`. Since this might confuse folks trying to read this data from storage in the future, I've updated the code to correctly use to `half_val` field for these cases. Note that there's no real size advantage from doing this, since both the `half_val` and `float_val` fields are 32 bits long.

Test Plan:
Added a parameterized unit test that tests encoding tensors with `torch.half`, `torch.float16`, and `torch.bfloat16` data types.

# Before this change
The test fails with an `ValueError` due to the incorrect enum label:
```
======================================================================
ERROR: test_bfloat16_tensor_proto (test_tensorboard.TestTensorProtoSummary)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/users/jcarreiro/fbsource/buck-out/v2/gen/fbcode/f88b3f368c9334db/caffe2/test/__tensorboard__/tensorboard#link-tree/torch/testing/_internal/common_utils.py", line 2382, in wrapper
    method(*args, **kwargs)
  File "/data/users/jcarreiro/fbsource/buck-out/v2/gen/fbcode/f88b3f368c9334db/caffe2/test/__tensorboard__/tensorboard#link-tree/test_tensorboard.py", line 871, in test_bfloat16_tensor_proto
    tensor_proto(
  File "/data/users/jcarreiro/fbsource/buck-out/v2/gen/fbcode/f88b3f368c9334db/caffe2/test/__tensorboard__/tensorboard#link-tree/torch/utils/tensorboard/summary.py", line 400, in tensor_proto
    tensor_proto = TensorProto(**tensor_proto_args)
ValueError: unknown enum label "DT_BFLOAT"

To execute this test, run the following from the base repo dir:
     python test/__tensorboard__/tensorboard#link-tree/test_tensorboard.py -k test_bfloat16_tensor_proto

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

----------------------------------------------------------------------
```

# After this change
The test passes.

Reviewed By: tanvigupta17

Differential Revision: D48828958

Pull Request resolved: https://github.com/pytorch/pytorch/pull/108351
Approved by: https://github.com/hamzajzmati, https://github.com/XilunWu
2023-09-14 23:12:22 +00:00
..
_nvfuser Remove dynamo+nvfuser (#105789) 2023-08-08 22:29:32 +00:00
ao/sparsity add pruning method: Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration (#95689) 2023-08-02 16:24:42 +00:00
autograd Don't run test/autograd/test_fallback.py in parallel (#106866) 2023-08-10 00:26:23 +00:00
backends/xeon
benchmark_utils [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
bottleneck_test
cpp replace torch::make_unique with std::make_unique (#108866) 2023-09-14 20:52:26 +00:00
cpp_api_parity [BE]: Update ruff to 0.285 (#107519) 2023-08-22 23:16:38 +00:00
cpp_extensions Fix CPUFallback Mechinasm on TensorList Type (#105209) 2023-08-05 15:38:30 +00:00
custom_backend [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
custom_operator [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
distributed [HSDP] Updating HSDP test - test_hsdp_init_with_device_mesh (#109202) 2023-09-14 20:03:22 +00:00
distributions Reland: Remove remaining global set_default_dtype calls from tests (#108088) 2023-09-07 03:04:34 +00:00
dynamo Support SymBool input to torch.compile (#107850) 2023-09-14 21:34:31 +00:00
edge [torchgen] Rename executorch's RuntimeContext to KernelRuntimeContext (#104892) 2023-07-14 21:15:50 +00:00
error_messages [BE] Enable ruff's UP rules and autoformat dynamo / functorch and refs (#105432) 2023-07-19 13:48:44 +00:00
expect Add refs/decomps for dot/vdot (#108194) 2023-08-31 15:30:23 +00:00
export [export] Preserve shape dynamism for unused inputs. (#109239) 2023-09-14 07:43:36 +00:00
forward_backward_compatibility Flash Attention v2 (#105602) 2023-09-13 13:59:05 +00:00
functorch Remove expected test failures for cond (#108709) 2023-09-14 21:34:31 +00:00
fx Revert "split by tag (#108892)" 2023-09-14 22:43:03 +00:00
inductor Increase tolerances for baddbmm opinfo test (#109164) 2023-09-14 18:37:04 +00:00
jit [jit] Skip NNAPI, test_ivalue, CPU NNC tests in fbcode (#108937) 2023-09-11 22:42:30 +00:00
jit_hooks
lazy
mobile [BE] Enable ruff's UP rules in pyproject.toml (#105437) 2023-07-21 19:14:52 +00:00
nn LazyBatchNorm{1-3}d support dict&set (#109015) 2023-09-12 09:09:59 +00:00
onnx [ONNX] Remove deprecated functions (#107208) 2023-09-14 19:09:56 +00:00
onnx_caffe2 [ONNX] Remove deprecated functions (#107208) 2023-09-14 19:09:56 +00:00
optim Add capturable ASGD impl (#107857) 2023-09-07 06:30:30 +00:00
package [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
profiler Increased logging threshold for profiler matching (#108010) 2023-08-30 17:16:36 +00:00
quantization [quant][be] Remove test/quantization/pt2e/test_quantize_pt2e_fx.py (#108925) 2023-09-14 18:35:17 +00:00
scripts
test_img
torch_np Dispatch numpy.take_along_axis to torch.take_along_dim (#108880) 2023-09-13 23:13:09 +00:00
typing Improve type annotations for jit.script (#108782) 2023-09-13 19:20:25 +00:00
_test_bazel.py
allowlist_for_publicAPI.json Add _foreach_sign (#106343) 2023-08-01 22:33:34 +00:00
conftest.py
create_dummy_torchscript_model.py
delete.py
HowToWriteTestsUsingFileCheck.md
linear.py
load_torchscript_model.py
minioptest_failures_dict.json [optests] Changed failures_dict format to json; automatic update of failures_dict (#109110) 2023-09-13 13:24:15 +00:00
mkl_verbose.py
mkldnn_verbose.py
run_doctests.sh
run_test.py inductor/test_max_autotune serial in CI (#109209) 2023-09-13 17:04:43 +00:00
simulate_nccl_errors.py
test_ao_sparsity.py add pruning method: Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration (#95689) 2023-08-02 16:24:42 +00:00
test_autocast.py Allow float dtype when Autocast CPU Disabled (#107348) 2023-09-01 00:49:44 +00:00
test_autograd.py Update CopySlices to not internal assert when grad_output is undefined (#108353) 2023-09-11 16:26:05 +00:00
test_autograd_fallback.py Don't run test/autograd/test_fallback.py in parallel (#106866) 2023-08-10 00:26:23 +00:00
test_binary_ufuncs.py [BE]: Update ruff to 0.285 (#107519) 2023-08-22 23:16:38 +00:00
test_bundled_images.py
test_bundled_inputs.py
test_comparison_utils.py
test_compile_benchmark_util.py unify TEST_CUDA (#106685) 2023-08-10 09:01:36 +00:00
test_complex.py Don't fastpath conj copy when conj/neg bit mismatch (#108881) 2023-09-08 20:44:43 +00:00
test_content_store.py
test_cpp_api_parity.py Reland: Remove remaining global set_default_dtype calls from tests (#108088) 2023-09-07 03:04:34 +00:00
test_cpp_extensions_aot.py
test_cpp_extensions_jit.py [BE]: Update ruff to 0.285 (#107519) 2023-08-22 23:16:38 +00:00
test_cpp_extensions_open_device_registration.py unify TEST_CUDA (#106685) 2023-08-10 09:01:36 +00:00
test_cuda.py [CUDA][CUDA Graphs] Fix CUDAGraph::reset function (#108896) 2023-09-11 19:49:31 +00:00
test_cuda_expandable_segments.py [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
test_cuda_multigpu.py Revert "Fix test failure in TestCudaMultiGPU.test_cuda_device_memory_allocated (#105501)" 2023-07-24 14:18:44 +00:00
test_cuda_nvml_based_avail.py
test_cuda_primary_ctx.py
test_cuda_sanitizer.py unify TEST_CUDA (#106685) 2023-08-10 09:01:36 +00:00
test_cuda_trace.py unify TEST_CUDA (#106685) 2023-08-10 09:01:36 +00:00
test_custom_ops.py [dynamo] Add BACKEND_MATCH guard to detect and recompile when backend changes (#107337) 2023-09-14 15:49:30 +00:00
test_dataloader.py Update ruff to v0.0.286 (#108058) 2023-08-28 22:55:56 +00:00
test_datapipe.py Disable compile for massive data pipe test (#109063) 2023-09-12 00:15:52 +00:00
test_decomp.py Introduce Tensor overload to linspace and logspace (#104889) 2023-09-11 23:30:40 +00:00
test_deploy.py
test_determination.py
test_dispatch.py [BE]: Update ruff to 0.285 (#107519) 2023-08-22 23:16:38 +00:00
test_dlpack.py Add DLPack bool support (#108486) 2023-09-08 17:55:33 +00:00
test_dynamic_shapes.py remove redundant dynamic_dim (#107815) 2023-08-24 10:46:24 +00:00
test_expanded_weights.py
test_fake_tensor.py Flash Attention v2 (#105602) 2023-09-13 13:59:05 +00:00
test_flop_counter.py Flash Attention v2 (#105602) 2023-09-13 13:59:05 +00:00
test_foreach.py Refactor test_foreach.py (#107869) 2023-09-14 19:39:26 +00:00
test_function_schema.py
test_functional_autograd_benchmark.py
test_functional_optim.py Cudagraphs support for compiled optimizers (#107504) 2023-08-31 20:47:18 +00:00
test_functionalization.py reorder proxy / fake modes so they always run last (#104482) 2023-08-29 02:36:48 +00:00
test_functionalization_of_rng_ops.py
test_futures.py [Reland] Update mypy to 1.4.1 (#105227) 2023-07-15 20:30:20 +00:00
test_fx.py Remove expected test failures for cond (#108709) 2023-09-14 21:34:31 +00:00
test_fx_experimental.py Run translation validation on tracing error. (#106645) 2023-08-14 13:43:34 +00:00
test_fx_passes.py
test_fx_reinplace_pass.py
test_hub.py Default permissions for torch.hub downloads (#82869) 2023-08-24 15:48:24 +00:00
test_import_stats.py
test_indexing.py [Fix Bug] Cannot assign index like x[[1,2], :] = 2 when torch.use_deterministic_algorithms(True) to main (#105833) 2023-08-07 17:00:19 +00:00
test_itt.py
test_jit.py Reland: Remove remaining global set_default_dtype calls from tests (#108088) 2023-09-07 03:04:34 +00:00
test_jit_autocast.py
test_jit_cuda_fuser.py Revert "Remove remaining global set_default_dtype calls from tests (#107246)" 2023-08-25 19:34:55 +00:00
test_jit_disabled.py
test_jit_fuser.py [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
test_jit_fuser_legacy.py
test_jit_fuser_te.py [jit] Skip NNAPI, test_ivalue, CPU NNC tests in fbcode (#108937) 2023-09-11 22:42:30 +00:00
test_jit_legacy.py
test_jit_llga_fuser.py [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
test_jit_profiling.py
test_jit_simple.py
test_jit_string.py
test_jiterator.py
test_kernel_launch_checks.py
test_legacy_vmap.py
test_license.py
test_linalg.py Reland: Remove remaining global set_default_dtype calls from tests (#108088) 2023-09-07 03:04:34 +00:00
test_logging.py [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
test_masked.py
test_maskedtensor.py
test_matmul_cuda.py Add better error messaging to scaled_mm (#108454) 2023-09-07 21:26:47 +00:00
test_meta.py Revert "add Half support for BatchNorm on CPU (#102070)" 2023-09-14 16:13:34 +00:00
test_metal.py
test_mkl_verbose.py
test_mkldnn.py Add channels_last3d support for mkldnn conv and mkldnn deconv (#95271) 2023-08-30 02:53:30 +00:00
test_mkldnn_fusion.py Add channels_last3d support for mkldnn conv and mkldnn deconv (#95271) 2023-08-30 02:53:30 +00:00
test_mkldnn_verbose.py
test_mobile_optimizer.py [BE]: Update ruff to 0.285 (#107519) 2023-08-22 23:16:38 +00:00
test_model_dump.py
test_model_exports_to_core_aten.py [aten decomp] Update sdpa decom (#108371) 2023-09-03 15:17:08 +00:00
test_module_init.py Reland "Add nn.CircularPad{*}d for consistency + fix no_batch_dim support (#106148)" (#106632) 2023-08-07 20:10:25 +00:00
test_modules.py Add backward check for test_memory_format (#106104) 2023-08-25 18:11:54 +00:00
test_monitor.py
test_mps.py Revert "Add meta and OpInfo for _embedding_bag_dense_backward (#109211)" 2023-09-14 22:29:12 +00:00
test_multiprocessing.py
test_multiprocessing_spawn.py
test_namedtensor.py [BE] f-stringify torch/ and scripts (#105538) 2023-07-21 19:35:24 +00:00
test_namedtuple_return_api.py [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
test_native_functions.py
test_native_mha.py
test_nestedtensor.py Add NestedTensor python subclass (#108314) 2023-09-11 18:29:20 +00:00
test_nn.py Revert "add Half support for BatchNorm on CPU (#102070)" 2023-09-14 16:13:34 +00:00
test_nnapi.py [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
test_numba_integration.py
test_numpy_interop.py
test_nvfuser_frontend.py
test_openmp.py [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
test_ops.py Revert "Remove fixed skips (#108674)" 2023-09-12 08:04:32 +00:00
test_ops_fwd_gradients.py Reland: Remove remaining global set_default_dtype calls from tests (#108088) 2023-09-07 03:04:34 +00:00
test_ops_gradients.py Reland: Remove remaining global set_default_dtype calls from tests (#108088) 2023-09-07 03:04:34 +00:00
test_ops_jit.py Reland: Remove remaining global set_default_dtype calls from tests (#108088) 2023-09-07 03:04:34 +00:00
test_optim.py
test_out_dtype_op.py Allow registering decomps for HigherOrderOp; add decomp for out_dtype (#108080) 2023-08-31 03:15:38 +00:00
test_overrides.py Improve torch.cuda.amp type hints (#108630) 2023-09-08 06:06:25 +00:00
test_package.py
test_per_overload_api.py
test_prims.py Remove dynamo+nvfuser (#105789) 2023-08-08 22:29:32 +00:00
test_proxy_tensor.py Add SymInt support to torch.take_along_dim (#108879) 2023-09-13 23:13:09 +00:00
test_pruning_op.py NumPy support in torch.compile (#106211) 2023-08-11 00:39:32 +00:00
test_public_bindings.py [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
test_python_dispatch.py Allow adding extra dispatch keys to wrapper tensor subclass (#108808) 2023-09-08 18:46:09 +00:00
test_pytree.py Serialize pytree to json string (#106116) 2023-08-27 14:34:49 +00:00
test_quantization.py [quant][be] Remove test/quantization/pt2e/test_quantize_pt2e_fx.py (#108925) 2023-09-14 18:35:17 +00:00
test_reductions.py [BE]: Update ruff to 0.285 (#107519) 2023-08-22 23:16:38 +00:00
test_scatter_gather_ops.py [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
test_schema_check.py [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
test_segment_reductions.py [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
test_serialization.py When byteorder record is missing load as little endian by default (#108343) 2023-09-04 15:20:22 +00:00
test_set_default_mobile_cpu_allocator.py
test_shape_ops.py
test_show_pickle.py
test_sort_and_select.py [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
test_sparse.py fall back to eager on NotImplementedError (#107863) 2023-09-07 21:22:20 +00:00
test_sparse_csr.py update triton pin with needed inductor change (#107722) 2023-08-29 04:31:44 +00:00
test_sparse_semi_structured.py [core][sparse][pruning] cuSPARSELt Kernels and ops. (#107398) 2023-08-25 07:04:15 +00:00
test_spectral_ops.py decomposition spectral ops fixes (#108360) 2023-09-09 04:48:09 +00:00
test_stateless.py Throw error if stateless.functional_call called with nn.DataParallel (#107403) 2023-08-18 03:02:04 +00:00
test_static_runtime.py
test_subclass.py Allow storage() to work on python tensor subclasses, but error on future data accesses (#107417) 2023-08-22 15:25:31 +00:00
test_sympy_utils.py Replace sympy.solve with a new simplified one. (#105877) 2023-08-02 17:53:29 +00:00
test_tensor_creation_ops.py Reland: Remove remaining global set_default_dtype calls from tests (#108088) 2023-09-07 03:04:34 +00:00
test_tensorboard.py [tensorboard] Fix TensorBoard summary encoding for torch.bfloat16 tensors (#108351) 2023-09-14 23:12:22 +00:00
test_tensorexpr.py [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
test_tensorexpr_pybind.py [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
test_testing.py [Inductor CUTLASS backend] Step 1: Inductor config for cuda / cutlass, util functions. (#107802) 2023-09-12 17:44:32 +00:00
test_throughput_benchmark.py
test_torch.py Add memory overlap check to meta_copy_ (#108989) 2023-09-12 23:28:14 +00:00
test_transformers.py Flash Attention v2 (#105602) 2023-09-13 13:59:05 +00:00
test_type_hints.py
test_type_info.py Reland: Remove remaining global set_default_dtype calls from tests (#108088) 2023-09-07 03:04:34 +00:00
test_type_promotion.py [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00
test_typing.py
test_unary_ufuncs.py [BE]: Update ruff to 0.285 (#107519) 2023-08-22 23:16:38 +00:00
test_utils.py Add fast traceback utilities (#107358) 2023-08-18 19:05:54 +00:00
test_view_ops.py Add skipLazy marker for tests and use it for tests not working with LazyTensor (#107382) 2023-08-22 22:34:36 +00:00
test_vulkan.py
test_weak.py removing some redundant str splits (#106089) 2023-09-01 00:22:58 +00:00
test_xnnpack_integration.py [BE] Enable ruff's UP rules and autoformat test/ (#105434) 2023-07-19 20:36:06 +00:00