Basic reading and writing (numpy)

These are some examples on how to read and write TSDF data into and from a numpy array, using the tsdf library.

import tsdf

Load some data

# The file names
metadata_path = "data/example_meta.json"
binary_filename = "example_binary.bin"

# Multiple metadata files (one for each binary) are loaded into a dictionary
# mapping the binary file name to the metadata object
metadata_dict = tsdf.load_metadata_from_path(metadata_path)

# Retrieve the metadata object we want, using the name of the binary as key
metadata = metadata_dict[binary_filename]

# Load the data
data = tsdf.load_ndarray_from_binary(metadata)

# Print some info
print(f"Data type:\t {data.dtype}")
print(f"Data shape:\t {data.shape}")

Data type:	 int16
Data shape:	 (10, 3)

Perform basic data processing

# Perform an operation, resulting in a different data type
processed_data_1 = (data / 10).astype('float32')

# Print some info
print(f"Processed data type:\t {processed_data_1.dtype}")
print(f"Data shape:\t\t {processed_data_1.shape}")

Processed data type:	 float32
Data shape:		 (10, 3)

Write the processed data

Write the processed data in binary format. The call returns the corresponding metadata object.

# The new name of the file
output_bin_filename = "tmp_example_processed.bin"

# Write the data to a new binary file
processed_metadata_1 = tsdf.write_binary_file(
        "data",
        output_bin_filename,
        processed_data_1,
        metadata.get_plain_tsdf_dict_copy(),
    )

print(f"File written to data/{output_bin_filename}")

File written to data/tmp_example_processed.bin

Write the TSDF metadata file

# Write new metadata file
output_meta_filename = "tmp_example_processed_meta.json"
tsdf.write_metadata([processed_metadata_1], output_meta_filename)
print(f"File written to data/{output_meta_filename}")

File written to data/tmp_example_processed_meta.json

Write a metadata file that combines multiple binary files

# Preprocess the original data to generate another data source
processed_data_2 = (data * 1000).astype("int32")

# Adjust the metadata slightly
updated_metadata = metadata.get_plain_tsdf_dict_copy()
updated_metadata.pop("scale_factors")  # remove the 'scale_factors'

# Save the new binary file
output_bin_filename_2 = "tmp_example_processed_2.bin"
processed_metadata_2 = tsdf.write_binary_file(
    "data",
    output_bin_filename_2,
    processed_data_2,
    updated_metadata,
)
print(f"File written to data/{output_bin_filename_2}")

# Write a metadata file that combines the two binary files
output_meta_filename_2 = "tmp_example_processed_2_meta.json"
tsdf.write_metadata([processed_metadata_1, processed_metadata_2],
    output_meta_filename_2)
print(f"File written to data/{output_meta_filename_2}")

File written to data/tmp_example_processed_2.bin
File written to data/tmp_example_processed_2_meta.json