API Reference

Core prediction

Predicts the output of the model on the test data.

braindec.predict.image_to_labels(image, model_path, vocabulary, vocabulary_emb, prior_probability, topk=10, logit_scale=None, return_posterior_probability=False, device=None, model=None, **kwargs)

Predict the labels of an image using a pre-trained model.

braindec.predict.image_to_labels_hierarchical(image, model_path, vocabulary, vocabulary_emb, prior_probability, cognitiveatlas, topk=10, logit_scale=None, device=None, model=None, **kwargs)

Predict the label of an image.

braindec.predict.preprocess_image(image, standardize=False, data_dir=None, space='MNI152', density=None, image_emb_gene=None)

Preprocess the image.

Parameters:

image – Images

Embeddings

Code to determine embeddings for text and images.

class braindec.embedding.ImageEmbedding(standardize: bool = False, nilearn_dir: str = None, neuromaps_dir: str = None, atlas: str = 'difumo', dimension: int = 512, space: str = 'MNI152', density: str = None)

Bases: object

generate_embedding(images) → ndarray

Generate embedding for a single image.

Parameters:

image – Input image as a numpy array

Returns:

Numpy array containing the embedding

class braindec.embedding.TextEmbedding(model_name: str = 'BrainGPT/BrainGPT-7B-v0.2', max_length: int = None, batch_size: int = 1, device: str = None)

Bases: object

chunk_text(texts: str) → List[str]

Split text into chunks that respect the model’s token limit.

Parameters:

text – Input text to be chunked

Returns:

List of text chunks (tokenized dictionaries)

clear_device_cache()

Clear memory cache for the current device type.

generate_embedding(token_embeddings, attention_mask) → ndarray

Generate embedding from token embeddings.

Parameters:

• token_embeddings – Token embeddings

• attention_mask – Attention mask

Returns:

Numpy array containing the average embedding

get_token_embeddings(tokenized: dict) → ndarray

Get token embeddings for a single text chunk.

Parameters:

tokenized – Tokenized dictionary containing input_ids and attention_mask

Returns:

Numpy array containing the embedding

mean_pooling(token_embeddings, attention_mask)

process_text(text: List[str]) → ndarray

Process text by chunking and averaging token embeddings.

Parameters:

text – Batch of texts

Returns:

Averaged embedding vector for the entire batch of texts

Cognitive Atlas

class braindec.cogatlas.CognitiveAtlas(data_dir=None, task_snapshot=None, concept_snapshot=None, concept_to_task=None, concept_to_process=None, reduced_tasks=None)

Bases: object

get_concept_id_from_name(names)

get_concept_idx_from_names(names)

get_concept_idx_from_process_idx(process_idx)

get_concept_idx_from_task_idx(task_idx)

get_concept_names_from_idx(concept_idx)

get_process_idx_from_names(names)

get_process_names_from_idx(process_idx)

get_task_id_from_name(names)

get_task_idx_from_concept_idx(concept_idx)

get_task_idx_from_names(names)

get_task_names_from_idx(task_idx)

task_df

# Drops tasks with short definitions self.task_df = self.task_df.loc[self.task_df[“definition_text”].str.len() > 90] reduced_tasks = reduced_tasks.loc[

reduced_tasks[“task”].isin(self.task_df[“name”])

].reset_index(drop=True)

Data fetching

Download published braindec assets from OSF.

braindec.fetcher.download_asset(name, destination_root='.', overwrite=False, node_id='dsj56', timeout=60)

Download a named asset from the built-in manifest.

braindec.fetcher.download_bundle(name, destination_root='.', overwrite=False, node_id='dsj56', timeout=60)

Download a predefined bundle of assets.

braindec.fetcher.download_osf_file(file_id, destination_root='.', overwrite=False, use_materialized_path=True, destination=None, timeout=60)

Download a single OSF file by id.

braindec.fetcher.download_osf_folder(remote_path, destination_root='.', node_id='dsj56', provider='osfstorage', overwrite=False, timeout=60)

Download all files under a folder path from the published OSF project.

braindec.fetcher.download_osf_path(remote_path, destination_root='.', node_id='dsj56', provider='osfstorage', overwrite=False, timeout=60)

Download a published OSF file or folder by its remote path.

braindec.fetcher.get_available_assets()

Return the names of downloadable assets and bundles.

braindec.fetcher.get_cogatlas_concepts()

Fetch concept definitions from the Cognitive Atlas API.

braindec.fetcher.get_cogatlas_tasks()

Fetch task definitions from the Cognitive Atlas API.

braindec.fetcher.get_data_dir(data_dir=None)

Return the default braindec data directory without importing heavy modules.

braindec.fetcher.list_remote_assets(node_id='dsj56', remote_path='/', provider='osfstorage', timeout=60)

List files and folders under an OSF path.

Model

Model

class braindec.model.Bottleneck(input_size, hidden_layers=(1024, 512, 256), dropout=0.2)

Bases: Module

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class braindec.model.CLIP(embedding_dim, output_dim=512, dropout=0.1, logit_scale=np.float64(2.659260036932778), logit_bias=None)

Bases: Module

encode_image(image)

encode_text(text)

forward(image, text)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class braindec.model.Decoder(embedding_dim, output_dim, dropout)

Bases: Module

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class braindec.model.Encoder3D(channels=(16, 32, 64), input_channels=1, kernel_size=3, stride=1, padding=1)

Bases: Module

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class braindec.model.Flatten(*args: Any, **kwargs: Any)

Bases: Module

forward(input)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class braindec.model.ImageModel(output_dim, dropout)

Bases: Module

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class braindec.model.MRI3dCNN(batch_size=8, input_channels=1, num_classes=3, input_shape=(91, 109, 91), channels=(16, 32, 64), hidden_layers=(1024, 512, 256), kernel_size=3, stride=1, dropout=0.2)

Bases: Module

bottleneck

self.out = nn.Sequential(

nn.Linear(self.hidden_layers[-1], num_classes), nn.Sigmoid(),

)

compute_output_shape(input_shape)

Dynamically compute the output shape.

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class braindec.model.ProjectionHead(embedding_dim, output_dim, dropout)

Bases: Module

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class braindec.model.ResidualHead(dim, dropout)

Bases: Module

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class braindec.model.TextModel(embedding_dim, output_dim, dropout)

Bases: Module

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

braindec.model.build_model(model_path, device)

Build model from a saved model file.

braindec.model.count_parameters(model)

Utilities

Miscellaneous functions used for analyses.

braindec.utils.get_data_dir(data_dir=None)

Get path to gradec data directory.

Parameters:

data_dir (str, optional) – Path to use as data directory. If not specified, will check for environmental variable ‘GRADEC_DATA’; if that is not set, will use ~/gradec-data instead. Default: None

Returns:

data_dir – Path to use as data directory

Return type:

str

Notes

Taken from Neuromaps. https://github.com/netneurolab/neuromaps/blob/abf5a5c3d3d011d644b56ea5c6a3953cedd80b37/ neuromaps/datasets/utils.py#LL91C1-L115C20

braindec.utils.images_have_same_fov(img, reference_img)

Return whether two Niimg-like objects share shape and affine.