Incremental CSV and TSV reader producing CompositeArray values. More...

#include <DelimitedTextReader.hpp>

Inheritance diagram for MayaFlux::IO::DelimitedTextReader:

Collaboration diagram for MayaFlux::IO::DelimitedTextReader:

Classes
struct	ParsedField

Public Member Functions
bool	can_read (const std::string &filepath) const override
	Check for the .csv or .tsv extension.

void	close () override
	Close the input and release its active layout.

	DelimitedTextReader ()=default

std::vector< uint64_t >	get_dimension_sizes () const override
	Total element count is not known without scanning the file.

std::optional< Kakshya::CompositeLayout >	get_layout () const override
	Copy the layout established by open(), if available.

std::optional< FileMetadata >	get_metadata () const override
	File metadata and ordered field names, when open.

size_t	get_num_dimensions () const override
	Delimited data has one element axis.

uint64_t	get_preferred_chunk_size () const override
	Suggested number of records per bounded read.

std::vector< uint64_t >	get_read_position () const override
	Zero-based index of the next data record.

std::vector< FileRegion >	get_regions () const override
	Delimited text does not declare semantic file regions.

std::vector< std::string >	get_supported_extensions () const override
	Supported file extensions without dots.

bool	is_open () const override
	Whether a file is currently open.

bool	open (const std::string &filepath, FileReadOptions options=FileReadOptions::ALL) override
	Open a CSV or TSV file and establish its Composite layout.

std::optional< Kakshya::CompositeArray >	read_composite () override
	Consume all remaining records into one array.

std::optional< Kakshya::CompositeArray >	read_next (size_t max_elements) override
	Consume a bounded number of records.

bool	seek (const std::vector< uint64_t > &position) override
	Seek to a zero-based data record by replaying from the first.

bool	set_delimiter (char delimiter) noexcept
	Override the delimiter chosen from .csv or .tsv.

bool	set_has_header (bool enabled) noexcept
	Select whether the first record supplies field names.

bool	set_layout (Kakshya::CompositeLayout layout)
	Set an exact field layout before opening a file.

bool	supports_streaming () const override
	Bounded sequential reads are supported.

	~DelimitedTextReader () override

Public Member Functions inherited from MayaFlux::IO::CompositeReader
std::shared_ptr< Kakshya::CompositeContainer >	create_composite_container (size_t batch_size=1)
	Read remaining elements into a new CompositeContainer.

std::shared_ptr< Kakshya::SignalSourceContainer >	create_container () override
	CompositeContainer is not a SignalSourceContainer.

std::type_index	get_container_type () const override
	Get the container type this reader creates.

std::type_index	get_data_type () const override
	Get the data type this reader produces.

std::string	get_last_error () const override
	Get the last error message.

bool	load_into_container (std::shared_ptr< Kakshya::SignalSourceContainer >) override
	CompositeContainer is not a SignalSourceContainer.

std::vector< Kakshya::DataVariant >	read_all () override
	Composite data is not a DataVariant channel vector.

std::optional< Kakshya::CompositeArray >	read_composite_region (const FileRegion &region)
	Read an inclusive one-dimensional file region.

std::vector< Kakshya::DataVariant >	read_region (const FileRegion &) override
	Use read_composite_region() for a schema-bearing result.

	~CompositeReader () override=default

Public Member Functions inherited from MayaFlux::IO::FileReader
virtual	~FileReader ()=default

Private Member Functions
bool	append_record (Kakshya::CompositeArray &array, const std::vector< ParsedField > &fields)

bool	read_record (std::vector< ParsedField > &fields, bool &at_end)

Private Attributes
char	m_active_delimiter {}

bool	m_at_end {}

std::streampos	m_data_start {}

char	m_delimiter {}

std::ifstream	m_file

std::string	m_filepath

bool	m_has_header { true }

std::optional< Kakshya::CompositeLayout >	m_layout

std::optional< Kakshya::CompositeLayout >	m_requested_layout

uint64_t	m_row_position {}

Additional Inherited Members
Static Public Member Functions inherited from MayaFlux::IO::FileReader
static std::string	resolve_path (const std::string &filepath)
	Resolve a filepath against the project source root if not found as-is.

Static Protected Member Functions inherited from MayaFlux::IO::FileReader
static std::unordered_map< std::string, Kakshya::RegionGroup >	regions_to_groups (const std::vector< FileRegion > &regions)
	Convert file regions to region groups.

Protected Attributes inherited from MayaFlux::IO::CompositeReader
std::string	m_last_error

Detailed Description

Incremental CSV and TSV reader producing CompositeArray values.

A header row supplies field names by default. Without a supplied layout, every field is stored as UTF-8 text, preserving values without speculative numeric inference. Supply a CompositeLayout before open() to select exact arithmetic field types; header names must then match its field order. Empty numeric cells and missing trailing cells remain absent. An unquoted empty text cell is absent; a quoted empty cell is present. char, signed char, and unsigned char fields parse as small integers via from_chars, the same path as every other arithmetic type; a cell must contain a number such as "65", not a literal character such as "A".

Usage:

CompositeLayout layout;
layout.add_field<std::string>("name");
layout.add_field<double>("score");
DelimitedTextReader reader;
 
reader.set_layout(std::move(layout));
if (reader.open("scores.csv")) {
    while (auto batch = reader.read_next(512)) {
        if (batch->empty())
            break;
    }
}

CSV and TSV share the same quoting rules. The delimiter defaults to the file extension and can be overridden before open().

Definition at line 42 of file DelimitedTextReader.hpp.

The documentation for this class was generated from the following files:

Classes

Public Member Functions

Private Member Functions

Private Attributes

Additional Inherited Members

Detailed Description