Virtual File System
Last updated
Last updated
Assoc. Prof. Wiroon Sriborrirux, Founder of Advance Innovation Center (AIC) and Bangsaen Design House (BDH), Electrical Engineering Department, Faculty of Engineering, Burapha University
In early embedded systems, the data that needed to be stored was relatively small and the data type was relatively simple. The method of directly writing data to the specified address in the storage device was often used to store data. However, with the development of embedded device functions, the data that needs to be stored is increasing and becoming more and more complex. At this time, it becomes very cumbersome and difficult to use the old method to store and manage data. Therefore, we need a new data management method to simplify the organization of stored data. This method is the file system that we will introduce next.
The file system is an abstract data type that implements data storage, hierarchical organization, access, and retrieval. It is a mechanism for providing users with low-level data access. The basic unit of storage in a file system is usually a file, that is, data is organized in a file-by-file manner. When there are many files, there will be a large number of files that are difficult to classify and have duplicate names. A folder exists as a container for multiple files.
This chapter explains the RT-Thread file system and takes you to understand the architecture, functional features and usage of the RT-Thread virtual file system.
DFS is a virtual file system component provided by RT-Thread. Its full name is Device File System, which is a device virtual file system. The name of the file system uses a style similar to UNIX files and folders. The directory structure is shown in the following figure:
In RT-Thread DFS, the file system has a unified root directory, which /
is represented by . The f1.bin file in the root directory is /f1.bin
represented by , and the files in the 2018 directory f1.bin
are /data/2018/f1.bin
represented by . That is, the directory separator is /
, which is exactly the same as UNIX/Linux, but different from Windows (Windows operating system uses \
as the directory separator).
The main features of the RT-Thread DFS component are:
Provides a unified POSIX file and directory operation interface for applications: read, write, poll/select, etc.
It supports multiple types of file systems, such as FatFS, RomFS, DevFS, etc., and provides management of common files, device files, and network file descriptors.
Supports various types of storage devices, such as SD Card, SPI Flash, Nand Flash, etc.
The hierarchical architecture of DFS is shown in the figure below, which is mainly divided into POSIX interface layer, virtual file system layer and device abstraction layer.
POSIX stands for Portable Operating System Interface of UNIX (POSIX for short). The POSIX standard defines the interface standards that the operating system should provide to applications. It is the general name for a series of API standards defined by IEEE for software to run on various UNIX operating systems.
The POSIX standard is intended to achieve software portability at the source code level. In other words, a program written for a POSIX-compatible operating system should be compilable and executable on any other POSIX operating system (even if it comes from another manufacturer). RT-Thread supports the POSIX standard interface, so it is easy to port Linux/Unix programs to the RT-Thread operating system.
In Unix-like systems, ordinary files, device files, and network file descriptors are the same file descriptors. In the RT-Thread operating system, DFS is used to achieve this unification. With this unification of file descriptors, we can use the poll/select interface to uniformly poll these descriptors, which brings convenience to the implementation of program functions.
The poll/select interface can be used to block and simultaneously detect whether a group of I/O devices that support non-blocking have events (such as readable, writable, high-priority error output, error, etc.) until a device triggers an event or exceeds the specified waiting time. This mechanism can help the caller find the currently ready device and reduce the complexity of programming.
Users can register specific file systems to DFS, such as FatFS, RomFS, DevFS, etc. The following are some common file system types:
FatFS is a file system compatible with Microsoft FAT format developed specifically for small embedded devices. It is written in ANSI C and has good hardware independence and portability. It is the most commonly used file system type in RT-Thread.
The traditional RomFS file system is a simple, compact, read-only file system that does not support dynamic erasure and storage and stores data in sequence. It supports applications running in XIP (execute in place) mode, saving RAM space when the system is running.
Jffs2 file system is a log flash file system. It is mainly used for NOR flash memory, based on MTD driver layer, and has the following features: readable and writable, supports data compression, is a log file system based on hash table, provides crash/power-off safety protection, provides write balancing support, etc.
DevFS is the device file system. After enabling this function in the RT-Thread operating system, the devices in the system can be virtualized into files in the /dev folder, so that the devices can be operated using read, write and other interfaces in the same way as files.
NFS Network File System (NFS) is a technology for sharing files between different machines and different operating systems over the network. During the development and debugging phase of the operating system, this technology can be used to create an NFS-based root file system on the host and mount it on an embedded device, so that the content of the root file system can be easily modified.
UFFS is the abbreviation of Ultra-low-cost Flash File System. It is an open source file system developed by Chinese people and is specially designed for using Nand Flash in small memory environments such as embedded devices. Compared with the Yaffs file system commonly used in embedded systems, it has the advantages of less resource usage, fast startup speed, and free of charge.
The device abstraction layer abstracts physical devices such as SD Card, SPI Flash, and Nand Flash into devices that can be accessed by the file system. For example, the FAT file system requires that storage devices must be block device types.
Different file system types are implemented independently of storage device drivers. Therefore, the file system function can only be used correctly after the driver interface of the underlying storage device is connected with the file system.
The initialization process of the file system is generally divided into the following steps:
Initializes the DFS components.
Initialize a file system of a specific type.
Create a block device on the storage.
Format a block device.
Mount the block device to the DFS directory.
When a file system is no longer in use, it can be unmounted.
The initialization of the DFS component is completed by the dfs_init() function. The dfs_init() function will initialize the relevant resources required by DFS and create some key data structures. With these data structures, DFS can find a specific file system in the system and obtain the operation method for files in a specific storage device. If automatic initialization is enabled (enabled by default), this function will be called automatically.
After the DFS component is initialized, the specific type of file system to be used needs to be initialized, that is, the specific type of file system needs to be registered in DFS. The interface for registering a file system is as follows:
This function does not need to be called by the user, it will be called by the initialization function of different file systems, such as the initialization function of the elm-FAT file system elm_init()
. After the corresponding file system is enabled, if automatic initialization is enabled (enabled by default), the file system initialization function will also be automatically called.
elm_init()
The function will initialize the elm-FAT file system. This function will call dfs_register(
the function to register the elm-FAT file system to DFS. The file system registration process is shown in the figure below:
Because only block devices can be mounted on the file system, it is necessary to create the required block devices on the storage device. If the storage device is SPI Flash, you can use the "Serial Flash Universal Driver Library SFUD" component, which provides various SPI Flash drivers and abstracts SPI Flash into block devices for mounting. The process of registering a block device is shown in the following figure:
After registering the block device, you need to create a file system of the specified type on the block device, that is, format the file system. You can use dfs_mkfs()
the function to format the specified storage device and create a file system. The interface for formatting the file system is as follows:
The possible values of the file system type (fs_name) and the corresponding file systems are shown in the following table:
Taking the elm-FAT file system to format a block device as an example, the formatting process is shown in the following figure:
You can also use mkfs
the command to format the file system. The result of formatting the block device sd0 is as follows:
In RT-Thread, mounting means attaching a storage device to an existing path. To access files in a storage device, we must mount the partition where the file is located to an existing path, and then access the storage device through this path. The interface for mounting a file system is as follows:
If there is only one storage device, you can mount it directly to the root directory /
.
When a file system is no longer needed, it can be uninstalled. The interface for uninstalling a file system is as follows:
This section introduces the functions related to file operations. File operations are generally based on the file descriptor fd, as shown in the following figure:
To open or create a file, you can call the following open() function:
A file can be opened in multiple ways, and multiple opening modes can be specified at the same time. For example, if a file is opened in O_RDONLY and O_CREAT modes, then if the specified file does not exist, the file will be created first and then opened in read-only mode. The file opening modes are shown in the following table:
When you have finished using a file and no longer need to use it, you can use close()
the function to close the file, which close()
will write the data back to the disk and release the resources occupied by the file.
To read the file contents, use read()
the function:
This function will read len bytes of the file pointed to by the parameter fd into the memory pointed to by the buf pointer. In addition, the file's read and write position pointer will move with the bytes read.
To write data to a file, use write()
the function:
This function will write len bytes in the memory pointed to by the buf pointer to the file pointed to by the parameter fd. In addition, the file's read and write position pointers will move with the bytes written.
To rename a file, use rename()
the function:
This function will change the file name specified by the parameter old to the file name specified by the parameter new. If the file specified by new already exists, it will be overwritten.
To get the file status, use the following stat()
function:
To delete files in a specified directory, you can use unlink()
the function:
To synchronize all modified file data in memory to the storage device, you can use fsync()
the function:
To query the file system related information, you can use statfs()
the function:
To monitor whether an I/O device has an event, you can use select()
the function:
The interface can be used to select()
block and simultaneously detect whether a group of I/O devices that support non-blocking have events (such as readable, writable, high-priority error output, error, etc.) until a device triggers an event or exceeds the specified waiting time.
This section introduces the functions commonly used in directory management. The operations on directories are generally based on the directory address, as shown in the following figure:
To create a directory, use mkdir()
the function:
This function is used to create a directory or folder. The parameter path is the absolute path of the directory. The parameter mode is not enabled in the current version, so fill in the default parameter 0x777.
To delete a directory, use rmdir()
the function:
To open a directory, use opendir()
the function:
To close a directory, you can use closedir()
the function:
This function is used to close a directory and must opendir()
be used in conjunction with the function.
To read the directory, use readdir()
the function:
This function is used to read a directory. The parameter d is the directory stream pointer. In addition, each time a directory is read, the pointer position of the directory stream will automatically move back by 1 position.
To get the read position of the directory stream, use telldir()
the function:
The return value of this function records the current position of a directory stream. This return value represents the offset from the beginning of the directory file . You can use this value in subsequent seekdir()
function calls to reset the directory to the current position. In other words, telldir()
the function can seekdir()
be used in conjunction with the function to reset the reading position of the directory stream to the specified offset.
To set the location of the directory to be read next time, you can use seekdir()
the function:
This is used to set the reading position of the directory stream parameter d. When calling readdir(), reading starts from this new position.
To reset the directory stream's reading position to the beginning, use rewinddir()
the function:
This function can be used to set the current reading position of the directory stream d to the initial position of the directory stream.
The specific configuration path of the file system in menuconfig is as follows:
The configuration menu description and corresponding macro definition are shown in the following table:
By default, the RT-Thread operating system does not enable the relative path function in order to obtain a smaller memory footprint. When the relative path support option is not enabled, you should use absolute directories when using file and directory interfaces for operations (because the current working directory does not exist in the system at this time). If you need to use the current working directory and relative directories, you can enable the relative path function in the file system configuration item.
After the option [*] Using mount table for file system
is selected, the corresponding macro will be enabled RT_USING_DFS_MNTTABLE
to turn on the automatic mount table function. The automatic mount table mount_table[]
is provided by the user in the application code. The user needs to specify the device name, mount path, file system type, read-write flags and private data in the table. After that, the system will traverse the mount table to perform the mount. It should be noted that the mount table must end with {0} to determine the end of the table.
An example of the automatic mount table mount_table[]
is shown below, where mount_table[0]
the five members are dfs_mount()
the five parameters of the function, which means mounting the elm file system to the / path of the flash0 device, with rwflag being 0 and data being 0; mount_table[1]
and ending with {0} to determine the end of the table.
After enabling the elm-FatFs file system in menuconfig, you can further configure elm-FatFs. The configuration menu description and corresponding macro definitions are shown in the following table:
By default, FatFs file naming has the following disadvantages:
The file name (excluding the suffix) cannot be longer than 8 characters, and the suffix cannot be longer than 3 characters. The file name and suffix will be truncated if they exceed the limit.
File names are not case sensitive (displayed as uppercase letters)
If you need to support long file names, you need to turn on the long file name support option. The long file name submenu description is as follows:
When long file name support is enabled, you can set the file name encoding. RT-Thread/FatFs uses 437 encoding (American English) by default. If you need to store Chinese file names, you can use 936 encoding (GBK encoding). 936 encoding requires a font library of about 180KB. If you only use English characters as files, it is recommended to use 437 encoding (American English), which can save 180KB of Flash space.
The file encodings supported by FatFs are as follows:
Specify the internal sector size of FatFs, which needs to be greater than or equal to the sector size of the actual hardware driver. For example, if the sector size of a spi flash chip is 4096 bytes, the above macro needs to be modified to 4096, otherwise the array will be out of bounds when FatFs reads data from the driver, causing the system to crash (the new version will give a warning message when the system is executed).
Generally, the sector size of Flash devices can be set to 4096, and the sector size of common TF cards and SD cards is set to 512.
FatFs takes into full consideration the safety of multi-threaded reading and writing. When reading and writing FatFs in multiple threads, in order to avoid the problem of reentry, the above macro needs to be turned on. If the system has only one thread operating the file system and there will be no reentry problem, this function can be turned off to save resources.
FatFs itself supports a lot of configuration options and is very flexible. The following file is the configuration file of FatFs, which can be modified to customize FatFs.
After the file system is mounted successfully, you can operate files and directories. The commonly used FinSH commands for file system operations are shown in the following table:
Use ls
the command to view the current directory information. The running results are as follows:
Use mkdir
the command to create a folder. The running result is as follows:
Use echo
the command to output the input string to the specified output location. The running result is as follows:
Use cat
the command to view the file contents. The running results are as follows:
Use rm
the command to delete folders or files. The running results are as follows:
After the file system works properly, you can run the application example. In this sample code, you will first use open()
the function to create a file text.txt, use the write() function to write a string to the file “RT-Thread Programmer!\n”
, and then close the file. Use the function again open()
to open the text.txt file, read out the content and print it out, and finally close the file.
The sample code is as follows:
The sample code in this section shows how to modify the file name. The program will create a function to operate the file rename_sample()
and export it to the msh command list. This function will call rename()
the function to rename the file named text.txt to text1.txt. The sample code is as follows:
Run this example in the FinSH console, and the results are as follows:
In the example display process, we first use the echo command to create a file named text.txt, and then run the sample code to change the file name of the text.txt file to text1.txt.
The sample code in this section shows how to get the file status. The program will create a function to operate the file stat_sample()
and export it to the msh command list. This function will call stat()
the function to get the file size information of the text.txt file. The sample code is as follows: