Skip to content

Features and confounds

Most hypline workflows use the command-line interface. Hypline also exposes a small set of functions directly from the hypline package for reading and writing its Parquet files.

from hypline import (
    read_feature,
    read_feature_metadata,
    read_confound,
    read_confound_metadata,
    save_feature,
    save_confound,
)

Reading and writing work slightly differently:

  • Saves are entity-based. You provide bids_root and the relevant BIDS entities (dyad, feat/conf, run, …). Hypline constructs the canonical output path for you, so that the file lands where downstream commands expect it.
  • Reads are path-based. You provide the path of an existing Parquet file.

Both operations validate the Hypline dataset layout and the expected file format. Invalid paths, metadata, or DataFrames raise an error instead of producing a file that hypline cannot read.

Fitted encoding models and evaluation results use separate loaders under hypline.encoding. See Encoding results.

Save a custom feature

The CLI can generate phonemic, semantic, spectral, and syntactic features. To use another set of features (e.g., prosody), you can construct the feature DataFrame yourself and save it with save_feature.

A feature DataFrame requires two columns:

  • start_time: the time in seconds from the beginning of the stimulus
  • feature: an equal-width feature vector for that time point

Follow the same start_time convention as hypline's generated features. See the feature file format.

import polars as pl
from hypline import save_feature

df = pl.DataFrame(
    {
        "start_time": [0.0, 0.48, 0.91],
        "feature": [[0.1, 0.2, 0.3], [0.0, 0.5, 0.1], [0.4, 0.4, 0.2]],
    }
)

path = save_feature(
    df,
    bids_root="data/",
    dyad="040",
    ses="1",
    feat="embed",
    task="conv",
    run="1",
)

This writes data/features/dyad-040/ses-1/embed/dyad-040_ses-1_task-conv_run-1_feat-embed.parquet. The saved file is now part of the hypline dataset, and downstream commands can locate it using the feature name embed.

Use desc="..." to save a variant in its own embed-<desc>/ subdirectory. Use metadata={...} to add custom keys to the Parquet footer.

Save a custom confound

save_confound is the corresponding function for custom confounds. Unlike a feature, a confound is regressed from the BOLD signal and must therefore contain one row per fMRI volume.

A confound DataFrame requires two columns:

  • start_time: the time in seconds from the beginning of the stimulus
  • confound: an equal-width confound vector for that time point

The start_time values must begin at 0.0 and advance by the run's repetition time. Mark your TR explicitly with repetition_time. tr_method records how the confound was converted to one row per fMRI volume (e.g., "mean" if you averaged signals within each TR), and is required. You can put None if it's not applicable (i.e., your confound was already one row per TR). For each confound, every run must have the same tr_method info. Otherwise, encoding train will complain about inconsistent metadata.

import polars as pl
from hypline import save_confound

df = pl.DataFrame(
    {
        "start_time": [0.0, 1.5, 3.0],
        "confound": [[0.9, 0.01, 0.03], [0.0, 0.2, 0.5], [0.1, 0.2, 0.6]],
    }
)

path = save_confound(
    df,
    bids_root="data/",
    dyad="040",
    ses="1",
    conf="newconf",
    task="conv",
    run="1",
    repetition_time=1.5,
    tr_method="mean",
)

Read data and metadata

Load the complete feature table with read_feature:

from hypline import read_feature

df = read_feature(path)

To inspect the Parquet footer without loading the feature values, use read_feature_metadata:

from hypline import read_feature_metadata

meta = read_feature_metadata(path)   # feature_name, feature_dim, hypline_version, …

The corresponding confound functions are read_confound and read_confound_metadata.

Read operations validate the path and cross-check its entities against the Parquet metadata. A file that passes read_feature or read_confound therefore satisfies hypline's feature- or confound-file validation.

API reference

Writing

hypline.save_feature

save_feature(df: DataFrame, *, bids_root: str | Path, dyad: str, feat: str, desc: str | None = None, metadata: dict[str, Any] | None = None, **entities: str) -> Path

Save a feature DataFrame to its canonical layout location.

Parameters:

Name Type Description Default
df DataFrame

Must contain start_time (numeric, source-relative seconds) and feature (Array or List of equal width per row). The feature column is normalized to fixed-width Array(Float64) on write; other columns are preserved.

required
bids_root str | Path

Project root; the file lands under <bids_root>/features/dyad-<dyad>/[ses-<ses>/]<feat>[-<desc>]/.

required
dyad str

Required dyad and feature-kind labels. Features describe the shared conversation, so they are keyed by dyad, not subject.

required
feat str

Required dyad and feature-kind labels. Features describe the shared conversation, so they are keyed by dyad, not subject.

required
desc str | None

Optional variant tag landing as a desc-<desc> entity, placing the file under a <feat>-<desc>/ subdirectory. Omit for the canonical (variant-free) feature.

None
metadata dict[str, Any] | None

Extra keys merged into the Parquet footer's hypline blob. The reserved keys feature_name, feature_dim, and hypline_version are auto-injected and must not be supplied. Keys prefixed with _ are exempt from cross-file equality checks at encoding time.

None
**entities str

Additional BIDS entities (ses, task, run, custom descriptors). Entity ordering and validation are handled by BIDSPath.

{}

Returns:

Type Description
Path

Resolved output path.

Raises:

Type Description
ValueError

If required columns are missing, feature widths are ragged, or metadata supplies a reserved key.

hypline.save_confound

save_confound(df: DataFrame, *, bids_root: str | Path, dyad: str, conf: str, repetition_time: float, tr_method: str | None, desc: str | None = None, metadata: dict[str, Any] | None = None, **entities: str) -> Path

Save a confound DataFrame to its canonical layout location.

Parameters:

Name Type Description Default
df DataFrame

Must contain start_time (numeric, beginning at 0.0 with intervals equal to repetition_time) and confound (Array or List of equal width per row). The confound column is normalized to fixed-width Array(Float64) on write.

required
bids_root str | Path

Project root; the file lands under <bids_root>/confounds/dyad-<dyad>/[ses-<ses>/]<conf>[-<desc>]/.

required
dyad str

Required dyad and confound-kind labels. Confounds describe the shared conversation, so they are keyed by dyad, not subject.

required
conf str

Required dyad and confound-kind labels. Confounds describe the shared conversation, so they are keyed by dyad, not subject.

required
repetition_time float

TR of the target BOLD acquisition, in seconds. Must be passed explicitly: a single-row DataFrame carries no spacing, and inferring TR from row spacing would silently disagree with the BOLD's true TR.

required
tr_method str | None

Free-form label for how TR-aligned rows were produced (downsampling/upsampling method, or a marker for native-TR computation). Pass None if not applicable. Recorded verbatim; must be equal across files sharing the same (conf, desc) pair for downstream consistency checks to pass.

required
desc str | None

Names which derivation of the kind's source this file holds, landing as a desc-<desc> entity under a <conf>-<desc>/ subdirectory. Omit for the kind's unnamed default derivation.

None
metadata dict[str, Any] | None

Extra keys merged into the Parquet footer's hypline blob. Reserved keys (confound_kind, confound_variant, tr_method, repetition_time, n_trs, confound_dim, hypline_version) are auto-injected and must not be supplied. Keys prefixed with _ are exempt from cross-file equality checks.

None
**entities str

Additional BIDS entities (ses, task, run, custom descriptors).

{}

Returns:

Type Description
Path

Resolved output path.

Raises:

Type Description
ValueError

If required columns are missing, start_time is not TR-aligned, confound widths are ragged, or metadata supplies a reserved key.

Reading

hypline.read_feature

read_feature(path: str | Path) -> pl.DataFrame

Read a feature DataFrame from a hypline-format Parquet file.

Validates that the path carries a feat entity, .parquet extension, and no BIDS suffix; checks the footer's feature_name matches the feat entity; normalizes the feature column to fixed-width Array(Float64).

Parameters:

Name Type Description Default
path str | Path

Path to a hypline feature Parquet file.

required

Returns:

Type Description
DataFrame

The stored feature DataFrame.

Raises:

Type Description
ValueError

If the path is not a valid hypline feature path, the footer lacks a hypline metadata blob, or feature_name disagrees with the path entity.

hypline.read_confound

read_confound(path: str | Path) -> pl.DataFrame

Read a confound DataFrame from a hypline-format Parquet file.

Validates that the path carries a conf entity, .parquet extension, and no BIDS suffix; checks that the footer's confound_kind and confound_variant match the path's conf and desc entities; normalizes the confound column to fixed-width Array(Float64).

Parameters:

Name Type Description Default
path str | Path

Path to a hypline confound Parquet file.

required

Returns:

Type Description
DataFrame

The stored confound DataFrame.

Raises:

Type Description
ValueError

If the path is not a valid hypline confound path, the footer lacks a hypline metadata blob, or confound_kind/confound_variant disagrees with the path entities.

Metadata

hypline.read_feature_metadata

read_feature_metadata(path: str | Path) -> dict[str, Any]

Read footer metadata from a hypline feature file without loading data.

Parameters:

Name Type Description Default
path str | Path

Path to a hypline feature Parquet file.

required

Returns:

Type Description
dict

The hypline JSON blob from the Parquet footer, including the auto-injected feature_name, feature_dim, and hypline_version keys alongside any caller-supplied keys.

Raises:

Type Description
ValueError

Same conditions as read_feature.

hypline.read_confound_metadata

read_confound_metadata(path: str | Path) -> dict[str, Any]

Read footer metadata from a hypline confound file without loading data.

Parameters:

Name Type Description Default
path str | Path

Path to a hypline confound Parquet file.

required

Returns:

Type Description
dict

The hypline JSON blob from the Parquet footer, including the auto-injected confound_kind, confound_variant, tr_method, repetition_time, n_trs, confound_dim, and hypline_version keys alongside any caller-supplied keys.

Raises:

Type Description
ValueError

Same conditions as read_confound.