Output Saving

This package provides functions for saving data to disk in convenient formats, supporting both JSON and HDF5. The output of mutual_information can be efficiently saved to disk using these functions.

JSON Output

The write_json function saves data from a dictionary to a specified directory, creating individual JSON files for each entry. This format is human-readable and easily shareable, ideal for cases where data needs to be inspected or transferred without specialized tools.

Usage

Function: write_json
Input: A directory path and a dictionary.
Output: JSON files for each key-value pair in the dictionary.

HDF5 Output

The write_hdf5! function allows for efficient storage of complex data structures into an HDF5 file. This format is highly suitable for large datasets due to its support for hierarchical data organization and fast I/O operations.

Usage

Function: write_hdf5!
Input: An HDF5 group and either a dictionary or a DataFrame.
Output: Data is stored in the HDF5 format, with nested structures represented as groups.

API

PathWeightSampling.write_json — Function

Writes the contents of a dictionary to separate JSON files in a specified directory.

Arguments

path::AbstractString: The directory path where JSON files will be created.
val::AbstractDict: A dictionary with keys as filenames and values as data to be serialized.

Description

Creates the directory at path if it does not exist. Iterates over each key-value pair in val, writes each value to a separate JSON file named after the key in the specified directory.

source

PathWeightSampling.write_hdf5! — Function

Writes data from a dictionary or DataFrame to an HDF5 group.

Arguments

group: The HDF5 group where the data will be stored.
dict::AbstractDict: A dictionary containing data to write, with keys as names and values as data.
df::AbstractDataFrame: A DataFrame containing data to write, with column names as keys.

Description

The function write_hdf5! serializes data from either a dictionary or a DataFrame into the specified HDF5 group. For dictionaries, each key-value pair is processed, writing the value to the HDF5 group under the corresponding key name. For DataFrames, each column is written using the column name as the key.

Behavior

For simple types like strings and numbers, values are stored as attributes.
For numerical arrays, values are stored directly in the group.
For more complex types (dictionaries or DataFrames), new groups are created recursively.

Example

using HDF5, DataFrames

data = Dict("numbers" => [1, 2, 3], "info" => Dict("a" => 10, "b" => 20))
df = DataFrame(a = 1:3, b = 4:6)

h5open("data.h5", "w") do file
    write_hdf5!(file, data)
    write_hdf5!(file, df)
end

source