File System APIs

The FileSystem class provides a simple interface for interacting with the filesystem. It provides a singleton class which manages the filesystem and also includes the relevant POSIX, newlib, and C++ standard library headers providing access to the filesystem. It is a wrapper around the LittleFS library and can be configured using menuconfig to use a custom partition label.

It also provides some utility functions for interacting with the filesystem and performing operations such as getting the total, used, and free space on the filesystem, and listing files in a directory.

API Reference

Header File

Classes

class FileSystem : public espp::BaseComponent

File system class.

This class is a singleton and should be accessed via the get()

method. The class is responsible for mounting the file system and providing access to the file system. It is configured via the menuconfig system and will use the partition with the label specified in the menuconfig. The partition must be formatted with the LittleFS file system. The file system is mounted at the root directory of the partition such that all files will be stored under the path “/<partition_label>/”.

The class provides methods to get the amount of free, used and total space on the file system. It also provides a method to get a human readable string for a byte size.

See also

get_partition_label()

File System Info Example

    auto &fs = espp::FileSystem::get();
    // NOTE: partition label is configured by menuconfig and should match the
    //       partition label in the partition table (partitions.csv).
    // returns a const char*
    auto partition_label = fs.get_partition_label();
    // returns a std::string
    auto mount_point = fs.get_mount_point();
    // returns a std::filesystem::path
    auto root_path = fs.get_root_path();
    logger.info("Partition label: {}", partition_label);
    logger.info("Mount point:     {}", mount_point);
    logger.info("Root path:       {}", root_path.string());
    // human_readable returns a string with the size and unit, e.g. 1.2 MB
    auto total_space = fs.human_readable(fs.get_total_space());
    auto free_space = fs.human_readable(fs.get_free_space());
    auto used_space = fs.human_readable(fs.get_used_space());
    logger.info("Total space: {}", total_space);
    logger.info("Free space:  {}", free_space);
    logger.info("Used space:  {}", used_space);

File System POSIX / NEWLIB Example

    auto mount_point = espp::FileSystem::get_mount_point();
    const std::string sandbox = std::string(mount_point) + "/" + std::string(test_dir);
    struct stat st;
    // check that it exists - IT SHOULDN'T
    logger.info("Directory {} exists: {}", sandbox,
                stat(sandbox.c_str(), &st) == 0 && S_ISDIR(st.st_mode));

    // make a directory
    mkdir(sandbox.c_str(), 0755);
    logger.info("Created directory {}", sandbox);

    // check that it exists - IT SHOULD
    logger.info("Directory {} exists: {}", sandbox,
                stat(sandbox.c_str(), &st) == 0 && S_ISDIR(st.st_mode));

    // write to a file
    std::string file = sandbox + "/" + std::string(test_file);
    FILE *fp = fopen(file.c_str(), "w");
    if (fp == nullptr) {
      logger.error("Couldn't open {} for writing!", file);
    } else {
      fwrite(file_contents.data(), 1, file_contents.size(), fp);
      fclose(fp);
      logger.info("Wrote '{}' to {}", file_contents, file);
    }

    // get the file size
    stat(file.c_str(), &st);
    size_t file_size = st.st_size;
    logger.info("File '{}' is {}", file, espp::FileSystem::human_readable(file_size));

    // read from a file
    fp = fopen(file.c_str(), "r"); // NOTE: could use rb for binary
    if (fp == nullptr) {
      logger.error("Couldn't open {} for reading!", file);
    } else {
      // // alternative way to get the file size if you've already opened it
      // fseek(f, 0, SEEK_END); // go to end
      // size_t file_size = ftell(f); // another way to get the file size
      // fseek(f, 0, SEEK_SET); // go back to beginning

      std::vector<char> bytes;
      bytes.resize(file_size);
      fread(bytes.data(), 1, file_size, fp);
      fclose(fp);
      logger.info("Read bytes from file {}", bytes);
    }

    // rename the file
    std::string file2 = sandbox + "/test2.csv";
    // Check if destination file exists before renaming
    if (stat(file2.c_str(), &st) == 0) {
      // Delete it if it exists
      unlink(file2.c_str());
    }
    // Rename original file
    if (rename(file.c_str(), file2.c_str()) != 0) {
      logger.error("Could not rename {} to {}", file, file2);
    } else {
      logger.info("Renamed '{}' to '{}'", file, file2);
    }

    // make a subdirectory
    std::string sub = sandbox + "/" + std::string(sub_dir);
    mkdir(sub.c_str(), 0755);
    logger.info("Created subdirectory {}", sub);

    // make a file in the subdirectory
    std::string sub_file = sub + "/subfile.txt";
    FILE *sub_fp = fopen(sub_file.c_str(), "w");
    if (sub_fp == nullptr) {
      logger.error("Couldn't open {} for writing!", sub_file);
    } else {
      fwrite(file_contents.data(), 1, file_contents.size(), sub_fp);
      fclose(sub_fp);
      logger.info("Wrote '{}' to {}", file_contents, sub_file);
    }

    // list files in a directory
    auto &fs = espp::FileSystem::get();
    espp::FileSystem::ListConfig config;
    std::string directory_listing = fs.list_directory(sandbox, config);
    logger.info("Directory listing for {}:\n{}", sandbox, directory_listing);

    // list all files recursively
    config.recursive = true;
    auto root = fs.get_root_path();
    std::string root_listing = fs.list_directory(root, config);
    logger.info("Recursive directory listing for {}:\n{}", root.string(), root_listing);

    // get a lsit of files in a directory
    auto files = fs.get_files_in_path(sandbox);
    logger.info("Files in {}: ", sandbox);
    for (const auto &f : files) {
      logger.info("\t{}", f);
    }

    files = fs.get_files_in_path(root, true); // include directories
    logger.info("Files in {} (including directories): ", root.string());
    for (const auto &f : files) {
      logger.info("\t{}", f);
    }

    files = fs.get_files_in_path(root, true, true); // include directories, recursive
    logger.info("Files in {} (including directories, recursive): ", root.string());
    for (const auto &f : files) {
      logger.info("\t{}", f);
    }

    files = fs.get_files_in_path(root, false, true); // do not include directories, recursive
    logger.info("Files in {} (not including directories, recursive): ", root.string());
    for (const auto &f : files) {
      logger.info("\t{}", f);
    }

    // cleanup
    auto items = {sub_file, sub, file, file2, sandbox};
    for (auto &item : items) {
      // use stat to figure out if it exists
      auto code = stat(item.c_str(), &st);
      bool exists = code == 0;
      if (!exists) {
        logger.warn("Not removing '{}', it doesn't exist!", item);
        continue;
      }
      // and if it is a directory
      bool is_dir = S_ISDIR(st.st_mode);
      int ec = is_dir ? rmdir(item.c_str()) : unlink(item.c_str());
      if (ec) {
        logger.error("Could not remove {}, error code: {}", item, ec);
      } else {
        logger.info("Cleaned up {}", item);
      }
    }

File System Info std::filesystem Example

    // NOTE: use the overloads that take ec as parameter, else it will throw
    //       exception on error.
    std::error_code ec;
    namespace fs = std::filesystem;
    // NOTE: cannot use chdir on littlefs , so we need to always use absolute
    //       paths.
    const fs::path sandbox = espp::FileSystem::get().get_root_path() / fs::path{test_dir};

    // check that it exists - IT SHOULDN'T
    logger.info("Directory {} exists: {}", sandbox.string(), fs::exists(sandbox));

    // make a directory
    fs::create_directory(sandbox, ec);
    if (ec) {
      logger.error("Could not create directory {} - {}", sandbox.string(), ec.message());
    } else {
      logger.info("Created directory {}: {}", sandbox.string(), fs::exists(sandbox));
    }

    // check that it exists - IT SHOULD
    logger.info("Directory {} exists: {}", sandbox.string(), fs::exists(sandbox));

    fs::path file = sandbox / fs::path{test_file};

    // write to a file
    std::ofstream ofs(file);
    ofs << file_contents;
    ofs.close();
    ofs.flush();
    logger.info("Wrote '{}' to {}", file_contents, file.string());

    // Get file size
    size_t file_size = fs::file_size(file, ec);
    if (ec) {
      logger.error("Could not get file size of '{}' - {}", file.string(), ec.message());
    } else {
      logger.info("File '{}' has a file size of {}", file.string(),
                  espp::FileSystem::human_readable(file_size));
    }

    // read from a file
    std::ifstream ifs(file, std::ios::in | std::ios::binary | std::ios::ate); // at the end
    // ifstream::pos_type file_size = ifs.tellg(); // an alternate way to get size
    ifs.seekg(0, std::ios::beg);
    // read bytes
    std::vector<char> file_bytes(file_size);
    ifs.read(file_bytes.data(), file_size);
    // convert bytes to string_view
    std::string_view file_string(file_bytes.data(), file_size);
    logger.info("Read bytes from file: {}", file_bytes);
    logger.info("Read string from file: {}", file_string);
    ifs.close();

    // rename the file
    fs::path file2 = sandbox / "test2.csv";
    fs::rename(file, file2, ec);
    if (ec) {
      logger.error("Could not rename {} to {} - {}", file.string(), file2.string(), ec.message());
    } else {
      logger.info("Renamed {} to {}", file.string(), file2.string());
    }

    // make a subdirectory
    fs::path sub = sandbox / fs::path{sub_dir};
    fs::create_directory(sub, ec);
    if (ec) {
      logger.error("Could not create directory {} - {}", sub.string(), ec.message());
    } else {
      logger.info("Created subdirectory {}", sub.string());
    }

    // make a file in the subdirectory
    fs::path sub_file = sub / "subfile.txt";
    std::ofstream sub_ofs(sub_file);
    sub_ofs << file_contents;
    sub_ofs.close();
    sub_ofs.flush();
    logger.info("Wrote '{}' to {}", file_contents, sub_file.string());

    // list files in a directory
    // NOTE: if we're using ESP-IDF < 5.4, we can't use directory_iterator
#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 4, 0)
    logger.info("Directory iterator:");
#else
    logger.warn(
        "NOTE: directory_iterator is not implemented in esp-idf less than 5.4, the following "
        "listing will not work.");
#endif
    // NOTE: directory_iterator is not implemented in esp-idf right now :(
    // directory_iterator can be iterated using a range-for loop
    for (auto const &dir_entry : fs::recursive_directory_iterator{sandbox, ec}) {
      logger.info("\t{}", dir_entry.path().string());
    }
    if (ec) {
      logger.error("Could not iterate over directory '{}': {}", sandbox.string(), ec.message());
      logger.info("\tThis is expected since directory_iterator is not implemented in esp-idf.");
    }

    logger.info("Recursive directory listing:");
    auto &espp_fs = espp::FileSystem::get();
    auto files = espp_fs.get_files_in_path(sandbox, true, true);
    for (const auto &f : files) {
      logger.info("\t{}", f);
    }

    // attempt to delete an etry that doesn't exist
    fs::path nonexistent = sandbox / "does_not_exist.txt";
    if (!espp_fs.remove(nonexistent, ec)) {
      logger.info("Properly returned error when trying to remove nonexistent file {} - {}",
                  nonexistent.string(), ec.message());
      ec.clear();
    } else {
      logger.error("Removed nonexistent file {} - this should not happen!", nonexistent.string());
    }

    // cleanup, use convenience functions
    // NOTE: cannot use fs::remove since it seems POSIX remove() doesn't work
    // We'll use espp::FileSystem::remove, which works for both files and
    // directories, and will recursively remove all the contents of the
    // directory.
    espp_fs.set_log_level(espp::Logger::Verbosity::DEBUG);
    if (!espp_fs.remove(sandbox, ec)) {
      logger.error("Could not remove {}", sandbox.string());
    } else {
      logger.info("Cleaned up {}", sandbox.string());
    }

    // now list entries in root (recursively) after cleanup
    logger.info("Recursive directory listing after cleanup:");
    files = espp_fs.get_files_in_path(espp_fs.get_root_path(), true, true);
    for (const auto &f : files) {
      logger.info("\t{}", f);
    }

Public Functions

size_t get_free_space() const

Get the amount of free space on the file system.

Returns:

The amount of free space in bytes

size_t get_total_space() const

Get the total amount of space on the file system.

Returns:

The total amount of space in bytes

size_t get_used_space() const

Get the amount of used space on the file system.

Returns:

The amount of used space in bytes

std::string get_file_time_as_string(const std::filesystem::path &path) const

Get the time of a file as a string.

This method gets the time of a file as a string in the format “Jan 01 00:00”.

See also

file_time_to_string()

Parameters:

path – The path to the file

Returns:

The time of the file as a string

std::vector<std::filesystem::path> get_files_in_path(const std::filesystem::path &path, bool include_directories = false, bool recursive = false)

Get a vector of files in a directory.

This method returns a vector of paths to the files in a directory.

Parameters:

  • path – The path to the directory

  • include_directories – Whether to include directories in the output

  • recursive – Whether to include files in subdirectories

Returns:

A vector of paths to the files in the directory

bool remove(const std::filesystem::path &path, std::error_code &ec)

Completely remove a file or directory (including contents)

This method removes a file or directory and all of its contents. If the path is a directory, it will iterate over the contents and remove them recursively. If the path is a file, it will remove the file. If the path does not exist, it will return false.

Parameters:

  • path – The path to the file or directory

  • ec – The error code to set if an error occurs

Returns:

Whether the file or directory was successfully removed

bool remove_contents(const std::filesystem::path &path, std::error_code &ec)

Remove the contents of a directory, but not the directory itself.

This method removes the contents of a directory, but not the directory itself. If the path is not a directory, it will return false. If the path does not exist, it will return false. If the path is a directory, it will iterate over the contents and remove them recursively.

Parameters:

  • path – The path to the directory

  • ec – The error code to set if an error occurs

Returns:

Whether the contents of the directory were successfully removed

inline std::string list_directory(const std::filesystem::path &path, const ListConfig &config, const std::string &prefix = "")

List the contents of a directory.

This method lists the contents of a directory. It returns a string containing the contents of the directory. The contents are formatted according to the config. The config is a struct with boolean values for each of the fields to include in the output. The fields are:

  • type: The type of the file (directory, file, etc.)

  • permissions: The permissions of the file

  • number_of_links: The number of links to the file

  • owner: The owner of the file

  • group: The group of the file

  • size: The size of the file

  • date_time: The date and time of the file

  • recursive: Whether to list the contents of subdirectories

Parameters:

  • path – The path to the directory

  • config – The config for the output

  • prefix – The prefix to use for the output

Returns:

The contents of the directory

std::string list_directory(const std::string &path, const ListConfig &config, const std::string &prefix = "")

List the contents of a directory.

This method lists the contents of a directory. It returns a string containing the contents of the directory. The contents are formatted according to the config. The config is a struct with boolean values for each of the fields to include in the output. The fields are:

  • type: The type of the file (directory, file, etc.)

  • permissions: The permissions of the file

  • number_of_links: The number of links to the file

  • owner: The owner of the file

  • group: The group of the file

  • size: The size of the file

  • date_time: The date and time of the file

  • recursive: Whether to list the contents of subdirectories

Parameters:

  • path – The path to the directory

  • config – The config for the output

  • prefix – The prefix to use for the output

Returns:

The contents of the directory

std::string file_entry_string(const std::filesystem::path &path, const ListConfig &config, const std::string &prefix = "")

Get a file entry formatted as a string.

This method returns a string formatted as a file entry. The file entry contains the type, permissions, number of links, owner, group, size, date and time, and name of the file. The file entry is formatted according to the config. The config is a struct with boolean values for each of the fields to include in the output. The fields are:

  • type: The type of the file (directory, file, etc.)

  • permissions: The permissions of the file

  • number_of_links: The number of links to the file

  • owner: The owner of the file

  • group: The group of the file

  • size: The size of the file

  • date_time: The date and time of the file

    See also

    file_entry_string()

Parameters:

  • path – The path to the file

  • config – The config for the output

  • prefix – The prefix to use for the output

Returns:

The file entry as a string

inline const std::string &get_name() const

Get the name of the component

Note

This is the tag of the logger

Returns:

A const reference to the name of the component

inline void set_log_tag(const std::string_view &tag)

Set the tag for the logger

Parameters:

tag – The tag to use for the logger

inline espp::Logger::Verbosity get_log_level() const

Get the log level for the logger

See also

Logger::Verbosity

See also

Logger::set_verbosity

Returns:

The verbosity level of the logger

inline void set_log_level(espp::Logger::Verbosity level)

Set the log level for the logger

See also

Logger::Verbosity

See also

Logger::set_verbosity

Parameters:

level – The verbosity level to use for the logger

inline void set_log_verbosity(espp::Logger::Verbosity level)

Set the log verbosity for the logger

See also

set_log_level

See also

Logger::Verbosity

See also

Logger::set_verbosity

Note

This is a convenience method that calls set_log_level

Parameters:

level – The verbosity level to use for the logger

inline espp::Logger::Verbosity get_log_verbosity() const

Get the log verbosity for the logger

See also

get_log_level

See also

Logger::Verbosity

See also

Logger::get_verbosity

Note

This is a convenience method that calls get_log_level

Returns:

The verbosity level of the logger

inline void set_log_rate_limit(std::chrono::duration<float> rate_limit)

Set the rate limit for the logger

See also

Logger::set_rate_limit

Note

Only calls to the logger that have _rate_limit suffix will be rate limited

Parameters:

rate_limit – The rate limit to use for the logger

Public Static Functions

static inline void set_mount_as_read_only(bool read_only)

Set whether to mount the file system as read only.

Note

This only has an effect if called before the file system is mounted, i.e. before the first call to get()

Parameters:

read_only – Whether the file system is mounted as read only

static inline bool is_mount_as_read_only()

Get whether the file system is mounted as read only.

Returns:

Whether the file system is mounted as read only

static inline void set_grow_on_mount(bool grow_on_mount)

Set whether to grow the file system on mount.

Note

This only has an effect if called before the file system is mounted, i.e. before the first call to get()

Parameters:

grow_on_mount – Whether to grow the file system on mount

static inline bool is_grow_on_mount()

Get whether the file system was grown on mount.

Returns:

Whether the file system was grown on mount

static std::string human_readable(size_t bytes)

Get a human readable string for a byte size.

This method returns a human readable string for a byte size. It is copied from the example on the page: https://en.cppreference.com/w/cpp/filesystem/file_size

Parameters:

bytes – The byte size

Returns:

The human readable string

static inline const char *get_partition_label()

Get the partition label.

Returns:

The partition label

static std::string get_mount_point()

Get the mount point.

The mount point is the root directory of the file system. It is the root directory of the partition with the partition label.

See also

get_root_path()

Returns:

The mount point

static std::filesystem::path get_root_path()

Get the root path.

The root path is the root directory of the file system.

See also

get_mount_point()

Returns:

The root path

static std::string to_string(const std::filesystem::perms &permissions)

Convert file permissions to a string.

This method converts file permissions to a string in the format “rwxrwxrwx”.

Parameters:

permissions – The file permissions

Returns:

The file permissions as a string

static std::string to_string(time_t time)

Convert a time_t to a string.

This method converts a time_t to a string in the format “Jan 01 00:00”.

Parameters:

time – The time_t to convert

Returns:

The time as a string

static inline FileSystem &get()

Access the singleton instance of the file system.

Returns:

Reference to the file system instance

template<typename TP>
static inline std::time_t to_time_t(TP tp)

Function to convert a time_point to a time_t.

This function converts a time_point to a time_t. This function is needed because the standard library does not provide a function to convert a time_point to a time_t (until c++20 but support seems lacking on esp32). This function is taken from https://stackoverflow.com/a/61067330

Template Parameters:

TP – The type of the time_point.

Parameters:

tp – The time_point to convert.

Returns:

The time_t.

struct ListConfig

Config for listing the contents of a directory.

This struct is used to configure the output of the list_directory() method. It contains boolean values for each of the fields to include in the output.

Public Members

bool type = true

The type of the file (directory, file, etc.)

bool permissions = true

The permissions of the file.

bool number_of_links = true

The number of links to the file.

bool owner = true

The owner of the file.

bool group = true

The group of the file.

bool size = true

The size of the file.

bool date_time = true

The date and time of the file.

bool recursive = false

Whether to list the contents of subdirectories.