AUDIO Devices

Audio Introduction

Audio devices are a very important part of embedded systems, responsible for sampling and outputting audio data. Audio devices usually consist of a data bus interface, a control bus interface, an audio codec (Codec), a speaker, and a microphone, as shown in the following figure:

Embedded audio system components

Audio Device Features

The RT-Thread Audio device driver framework is the underlying part of the Audio framework, which is mainly responsible for the acquisition and output of native audio data, audio stream control, audio device management, volume adjustment, and abstraction of different hardware and Codecs.

• Interface: standard device interface (open/close/read/control).

• Synchronous mode access.

• Supports playback and recording.

• Support audio parameter management.

• Support volume adjustment.

Accessing Audio Devices

Find Audio Devices

The application obtains the device handle according to the Audio device name, and then can operate the Audio device. The device search function is as follows:

rt_device_t rt_device_find(const char* name);copymistakeCopy Success

parameter	describe
name	Audio Device Name
return	——
Device handle	If the corresponding device is found, the corresponding device handle will be returned.
RT_NULL	No corresponding device object was found

The usage examples are as follows:

#define SOUND_DEVICE_NAME    "sound0"    /* Audio 设备名称 */

static rt_device_t snd_dev;              /* Audio 设备句柄 */

/* 根据设备名称查找 Audio 设备，获取设备句柄 */
snd_dev = rt_device_find(SOUND_DEVICE_NAME);copymistakeCopy Success

Open Audio Device

Through the device handle, the application can open and close the device, and open the device through the following function:

rt_err_t rt_device_open(rt_device_t dev, rt_uint16_t oflags);copymistakeCopy Success

parameter	describe
dev	Device handle
oflags	Device mode flags
return	——
RT_EOK	Device opened successfully
- RT_EBUSY	If the device registration parameter includes the RT_DEVICE_FLAG_STANDALONE parameter, the device will not be allowed to be opened repeatedly.
-RT_EINVAL	Unsupported open parameters
Other error codes	Device open failed

The oflags parameter supports the following parameters:

#define RT_DEVICE_OFLAG_WRONLY      0x002     /* 标准设备的只写模式，对应 Audio 播放设备 */
#define RT_DEVICE_FLAG_RDONLY       0x001     /* 标准设备的只读模式，对应 Audio 录音设备 */copymistakeCopy Success

Audio devices are divided into two types: playback and recording. The playback device outputs audio data to the Codec, while the recording device reads the data. When in use, the playback device is identified by a write-only flag, and the recording device is identified by a read-only flag.

The following is an example of opening the Audio playback device:

rt_device_open(snd_dev, RT_DEVICE_OFLAG_WRONLY)copymistakeCopy Success

The following is an example of opening the Audio recording device:

rt_device_open(mic_dev, RT_DEVICE_FLAG_RDONLY)copymistakeCopy Success

Controlling Audio Devices

Through the command control word, the application can configure the Audio device through the following functions:

rt_err_t rt_device_control(rt_device_t dev, rt_uint8_t cmd, void* arg);copymistakeCopy Success

parameter	describe
dev	Device handle
cmd	Command control word, see below for details
arg	Control parameters, detailed description see below
return	——
RT_EOK	Function execution successful
-RT_ENOSYS	Execution failed, dev is empty
Other error codes	Execution failed

The cmd currently supports the following command control characters:

/* AUDIO command */
#define _AUDIO_CTL(a) (0x10 + a)

#define AUDIO_CTL_GETCAPS                   _AUDIO_CTL(1) /* 获取设备功能属性 */
#define AUDIO_CTL_CONFIGURE                 _AUDIO_CTL(2) /* 配置设备功能属性 */
copymistakeCopy Success

• The definition of the device function attribute structure is as follows

struct rt_audio_caps
{
    int main_type;                            /* 命令主类型 */
    int sub_type;                             /* 命令子类型 */

    union
    {
        rt_uint32_t mask;
        int     value;                       /* 参数值 */
        struct rt_audio_configure config;    /* 音频参数信息 */
    } udata;
};copymistakeCopy Success

Set the audio parameters for playback

Set the playback sampling rate, sampling channel, and sampling bit number.

struct rt_audio_caps caps;

caps.main_type               = AUDIO_TYPE_OUTPUT; /* 输出类型（播放设备 ）*/
caps.sub_type                = AUDIO_DSP_PARAM;   /* 设置所有音频参数信息 */
caps.udata.config.samplerate = 44100;             /* 采样率 */
caps.udata.config.channels   = 2;                 /* 采样通道 */
caps.udata.config.samplebits = 16;                /* 采样位数 */
rt_device_control(device, AUDIO_CTL_CONFIGURE, &caps);
copymistakeCopy Success

Set the master volume for playback

Sets the master volume for playback.

struct rt_audio_caps caps;

caps.main_type   = AUDIO_TYPE_MIXER;       /* 音量管理类型 */
caps.sub_type    = AUDIO_MIXER_VOLUME;     /* 设置播放的主音量 */
caps.udata.value = volume;                 /* 范围 0 ~ 100 */
rt_device_control(snd_dev, AUDIO_CTL_CONFIGURE, &caps);
copymistakeCopy Success

Set the audio parameter information of the recording

Set the recording sampling rate, sampling channel, and sampling bit number.

struct rt_audio_caps caps;

caps.main_type               = AUDIO_TYPE_INPUT;  /* 输入类型（录音设备 ）*/
caps.sub_type                = AUDIO_DSP_PARAM;   /* 设置所有音频参数信息 */
caps.udata.config.samplerate = 44100;             /* 采样率 */
caps.udata.config.channels   = 2;                 /* 采样通道 */
caps.udata.config.samplebits = 16;                /* 采样位数 */
rt_device_control(device, AUDIO_CTL_CONFIGURE, &caps);
copymistakeCopy Success

Set the master volume for recording

Set the master volume for recording.

struct rt_audio_caps caps;

caps.main_type = AUDIO_TYPE_MIXER;       /* 音量管理类型 */
caps.sub_type  = AUDIO_MIXER_MIC;        /* 设置录音的主音量 */
caps.udata.value = volume;               /* 范围 0 ~ 100 */
rt_device_control(player->device, AUDIO_CTL_CONFIGURE, &caps);
copymistakeCopy Success

Writing audio data

Writing data to the audio playback device can be done through the following function:

rt_size_t rt_device_write(rt_device_t dev, rt_off_t pos, const void* buffer, rt_size_t size);copymistakeCopy Success

parameter	describe
dev	Device handle
pos	Write data offset. This parameter is not used by audio devices.
buffer	Memory buffer pointer where the data to be written is placed
size	The size of the data to be written
return	——
The actual size of the data written	In bytes;

Calling this function will write the data in the buffer buffer to the device dev. The size of the written data is size. This function is a synchronous interface. The driver framework will first save the data to the buffer of the audio device. When the buffer is full, the function is blocked.

Reading audio data

The following function can be called to read the data received by the audio recording device:

rt_size_t rt_device_read(rt_device_t dev, rt_off_t pos, void* buffer, rt_size_t size);copymistakeCopy Success

parameter	describe
dev	Device handle
pos	Read data offset. This parameter is not used by the serial port device.
buffer	Buffer pointer. The read data will be saved in the buffer.
size	The size of the data to be read
return	——
The actual size of the data read	If it is a character device, the returned size is in bytes
0	You need to read the errno of the current thread to determine the error status

Call this function to read data of size from the audio recording device into the buffer. This function is a synchronous interface. When the data in the Pipe cache inside the driver framework is less than size, the function is blocked.

Turn off the audio device

When the application completes the serial port operation, it can close the audio device, which is done through the following function:

rt_err_t rt_device_close(rt_device_t dev);copymistakeCopy Success

parameter	describe
dev	Device handle
return	——
RT_EOK	Shut down the device successfully
-RT_ERROR	The device has been completely shut down. You cannot shut down the device again.
Other error codes	Failed to shut down the device

Closing the device interface and opening the device interface must be used in pairs. Each time you open the device, you must close it once. Only in this way can the device be completely closed. Otherwise, the device will still be in an open state.

Audio device usage examples

Audio devices are used for playback and recording, usually with the encoding and decoding of audio files. The following is an example of playing and recording wav files. The complete code can be obtained through the RT-Thread wavplayer software package .

Play

The main steps to play a piece of audio data are as follows:

1. First, find the Audio device and get the device handle.

2. Open the Audio device in write-only mode.

3. Set audio parameter information (sampling rate, channel, etc.).

4. Decode the audio file's data.

5. Write audio file data.

6. When playback is complete, turn off the device.

#include <rtthread.h>
#include <rtdevice.h>
#include <dfs_posix.h>

#define BUFSZ   1024
#define SOUND_DEVICE_NAME    "sound0"    /* Audio 设备名称 */
static rt_device_t snd_dev;              /* Audio 设备句柄 */

struct RIFF_HEADER_DEF
{
    char riff_id[4];     // 'R','I','F','F'
    uint32_t riff_size;
    char riff_format[4]; // 'W','A','V','E'
};

struct WAVE_FORMAT_DEF
{
    uint16_t FormatTag;
    uint16_t Channels;
    uint32_t SamplesPerSec;
    uint32_t AvgBytesPerSec;
    uint16_t BlockAlign;
    uint16_t BitsPerSample;
};

struct FMT_BLOCK_DEF
{
    char fmt_id[4];    // 'f','m','t',' '
    uint32_t fmt_size;
    struct WAVE_FORMAT_DEF wav_format;
};

struct DATA_BLOCK_DEF
{
    char data_id[4];     // 'R','I','F','F'
    uint32_t data_size;
};

struct wav_info
{
    struct RIFF_HEADER_DEF header;
    struct FMT_BLOCK_DEF   fmt_block;
    struct DATA_BLOCK_DEF  data_block;
};

int wavplay_sample(int argc, char **argv)
{
    int fd = -1;
    uint8_t *buffer = NULL;
    struct wav_info *info = NULL;
    struct rt_audio_caps caps = {0};

    if (argc != 2)
    {
        rt_kprintf("Usage:\n");
        rt_kprintf("wavplay_sample song.wav\n");
        return 0;
    }

    fd = open(argv[1], O_RDONLY);
    if (fd < 0)
    {
        rt_kprintf("open file failed!\n");
        goto __exit;
    }

    buffer = rt_malloc(BUFSZ);
    if (buffer == RT_NULL)
        goto __exit;

    info = (struct wav_info *) rt_malloc(sizeof * info);
    if (info == RT_NULL)
        goto __exit;

    if (read(fd, &(info->header), sizeof(struct RIFF_HEADER_DEF)) <= 0)
        goto __exit;
    if (read(fd, &(info->fmt_block),  sizeof(struct FMT_BLOCK_DEF)) <= 0)
        goto __exit;
    if (read(fd, &(info->data_block), sizeof(struct DATA_BLOCK_DEF)) <= 0)
        goto __exit;

    rt_kprintf("wav information:\n");
    rt_kprintf("samplerate %d\n", info->fmt_block.wav_format.SamplesPerSec);
    rt_kprintf("channel %d\n", info->fmt_block.wav_format.Channels);

    /* 根据设备名称查找 Audio 设备，获取设备句柄 */
    snd_dev = rt_device_find(SOUND_DEVICE_NAME);

    /* 以只写方式打开 Audio 播放设备 */
    rt_device_open(snd_dev, RT_DEVICE_OFLAG_WRONLY);

    /* 设置采样率、通道、采样位数等音频参数信息 */
    caps.main_type               = AUDIO_TYPE_OUTPUT;                           /* 输出类型（播放设备 ）*/
    caps.sub_type                = AUDIO_DSP_PARAM;                             /* 设置所有音频参数信息 */
    caps.udata.config.samplerate = info->fmt_block.wav_format.SamplesPerSec;    /* 采样率 */
    caps.udata.config.channels   = info->fmt_block.wav_format.Channels;         /* 采样通道 */
    caps.udata.config.samplebits = 16;                                          /* 采样位数 */
    rt_device_control(snd_dev, AUDIO_CTL_CONFIGURE, &caps);

    while (1)
    {
        int length;

        /* 从文件系统读取 wav 文件的音频数据 */
        length = read(fd, buffer, BUFSZ);

        if (length <= 0)
            break;

        /* 向 Audio 设备写入音频数据 */
        rt_device_write(snd_dev, 0, buffer, length);
    }

    /* 关闭 Audio 设备 */
    rt_device_close(snd_dev);

__exit:

    if (fd >= 0)
        close(fd);

    if (buffer)
        rt_free(buffer);

    if (info)
        rt_free(info);

    return 0;
}

MSH_CMD_EXPORT(wavplay_sample,  play wav file);
copymistakeCopy Success

recording

The main steps for recording a piece of audio data are as follows:

1. First, find the Audio device and get the device handle.

2. Open the Audio device in read-only mode.

3. Set audio parameter information (sampling rate, channel, etc.).

4. Read data from the audio device.

5. Process the read data, etc.

6. When the recording is complete, turn off the device.

#include <rtthread.h>
#include <rtdevice.h>
#include <dfs_posix.h>

#define RECORD_TIME_MS      5000
#define RECORD_SAMPLERATE   16000
#define RECORD_CHANNEL      2
#define RECORD_CHUNK_SZ     ((RECORD_SAMPLERATE * RECORD_CHANNEL * 2) * 20 / 1000)

#define SOUND_DEVICE_NAME    "mic0"      /* Audio 设备名称 */
static rt_device_t mic_dev;              /* Audio 设备句柄 */

struct wav_header
{
    char  riff_id[4];              /* "RIFF" */
    int   riff_datasize;           /* RIFF chunk data size,exclude riff_id[4] and riff_datasize,total - 8 */
    char  riff_type[4];            /* "WAVE" */
    char  fmt_id[4];               /* "fmt " */
    int   fmt_datasize;            /* fmt chunk data size,16 for pcm */
    short fmt_compression_code;    /* 1 for PCM */
    short fmt_channels;            /* 1(mono) or 2(stereo) */
    int   fmt_sample_rate;         /* samples per second */
    int   fmt_avg_bytes_per_sec;   /* sample_rate * channels * bit_per_sample / 8 */
    short fmt_block_align;         /* number bytes per sample, bit_per_sample * channels / 8 */
    short fmt_bit_per_sample;      /* bits of each sample(8,16,32). */
    char  data_id[4];              /* "data" */
    int   data_datasize;           /* data chunk size,pcm_size - 44 */
};

static void wavheader_init(struct wav_header *header, int sample_rate, int channels, int datasize)
{
    memcpy(header->riff_id, "RIFF", 4);
    header->riff_datasize = datasize + 44 - 8;
    memcpy(header->riff_type, "WAVE", 4);
    memcpy(header->fmt_id, "fmt ", 4);
    header->fmt_datasize = 16;
    header->fmt_compression_code = 1;
    header->fmt_channels = channels;
    header->fmt_sample_rate = sample_rate;
    header->fmt_bit_per_sample = 16;
    header->fmt_avg_bytes_per_sec = header->fmt_sample_rate * header->fmt_channels * header->fmt_bit_per_sample / 8;
    header->fmt_block_align = header->fmt_bit_per_sample * header->fmt_channels / 8;
    memcpy(header->data_id, "data", 4);
    header->data_datasize = datasize;
}

int wavrecord_sample(int argc, char **argv)
{
    int fd = -1;
    uint8_t *buffer = NULL;
    struct wav_header header;
    struct rt_audio_caps caps = {0};
    int length, total_length = 0;

    if (argc != 2)
    {
        rt_kprintf("Usage:\n");
        rt_kprintf("wavrecord_sample file.wav\n");
        return -1;
    }

    fd = open(argv[1], O_WRONLY | O_CREAT);
    if (fd < 0)
    {
        rt_kprintf("open file for recording failed!\n");
        return -1;
    }
    write(fd, &header, sizeof(struct wav_header));

    buffer = rt_malloc(RECORD_CHUNK_SZ);
    if (buffer == RT_NULL)
        goto __exit;

    /* 根据设备名称查找 Audio 设备，获取设备句柄 */
    mic_dev = rt_device_find(SOUND_DEVICE_NAME);
    if (mic_dev == RT_NULL)
        goto __exit;

    /* 以只读方式打开 Audio 录音设备 */
    rt_device_open(mic_dev, RT_DEVICE_OFLAG_RDONLY);

    /* 设置采样率、通道、采样位数等音频参数信息 */
    caps.main_type               = AUDIO_TYPE_INPUT;                            /* 输入类型（录音设备 ）*/
    caps.sub_type                = AUDIO_DSP_PARAM;                             /* 设置所有音频参数信息 */
    caps.udata.config.samplerate = RECORD_SAMPLERATE;                           /* 采样率 */
    caps.udata.config.channels   = RECORD_CHANNEL;                              /* 采样通道 */
    caps.udata.config.samplebits = 16;                                          /* 采样位数 */
    rt_device_control(mic_dev, AUDIO_CTL_CONFIGURE, &caps);

    while (1)
    {
        /* 从 Audio 设备中，读取 20ms 的音频数据  */
        length = rt_device_read(mic_dev, 0, buffer, RECORD_CHUNK_SZ);

        if (length)
        {
            /* 写入音频数据到到文件系统 */
            write(fd, buffer, length);
            total_length += length;
        }

        if ((total_length / RECORD_CHUNK_SZ) >  (RECORD_TIME_MS / 20))
            break;
    }

    /* 重新写入 wav 文件的头 */
    wavheader_init(&header, RECORD_SAMPLERATE, RECORD_CHANNEL, total_length);
    lseek(fd, 0, SEEK_SET);
    write(fd, &header, sizeof(struct wav_header));
    close(fd);

    /* 关闭 Audio 设备 */
    rt_device_close(mic_dev);

__exit:
    if (fd >= 0)
        close(fd);

    if (buffer)
        rt_free(buffer);

    return 0;
}
MSH_CMD_EXPORT(wavrecord_sample, record voice to a wav file);copymistakeCopy Success