pydoit_nb.serialization#

Serialization tools

This module includes a default yaml converter. This may be exactly what you need for your use case. If it isn’t, the code will help you see how to set up and customise a converter. This example, along with the cattrs docs, should help you get going. We mainly use cattrs.register_structure_hook_func() and cattrs.register_unstructure_hook_func() (see also here ), which aren’t as heavily documented, but are what we want. You may find the other examples in the docs on customisation more helpful.

One example of something we don’t support well is preservation of data types when structuring and unstructuring pint types. This is just a complicated problem to do with type hinting. For most applications, we have found that the default numpy data type is fine (everything becomes float64, which is generally fine).

One other clear example of something we don’t support here is structuring and unstructuring pandas objects as yaml. The reason is that we can’t see an easy way to get the round-tripping to work in edge cases related to index naming and data type preservation. There are ways to solve this problem, but they require more care and time than we can put in right now. As two suggestions for possible solutions: 1) use a back-end that isn’t yaml 2) store information about index types etc. in addition to the data frame (pandas itself doesn’t do this though, which makes us a bit nervous about how hard it could be to do in the general case, although specific use cases should be far more tractable and easy to test). If you’d like to discuss this more, please raise an issue.

write_config_in_config_bundle_to_disk#

write_config_in_config_bundle_to_disk(config_bundle, converter)[source]#

Write the configuration in a configuration bundle to disk

The configuration is written in the path specified by config_bundle.config_hydrated_path

Parameters:

config_bundle (pydoit_nb.typing.ConfigBundleLike[typing.TypeVar(U)]) – Configuration bundle to write to disk
converter (pydoit_nb.typing.Converter) – Object that can serialize the configuration bundle’s hydrated config

Returns:

pathlib.Path – Path in which config_bundle.config_hydrated was written

load_config_from_file#

load_config_from_file(config_file, target, converter)[source]#

Load configuration from file

Parameters:

config_file (pathlib.Path) – File from which to load configuration
target (type[typing.TypeVar(T)]) – Class to load
converter (pydoit_nb.typing.Converter) – Converter to use to convert from config_file’s contents to an instance of target

Returns:

typing.TypeVar(T) – Loaded instance of target

unstructure_np_array#

unstructure_np_array(arr)[source]#

Unstructure npt.ArrayLike

This simply converts it to a list so is probably not very fast. However, this is just an example so could easily be optimised for production use if needed.