Networks and chains
AbstractNN- Helferlein network typeAbstractChain- Helferlein chain type
Classifier- network with NLL lossRegressor- network with MSE sollVAE- variational autoencoder wrapper The VAE supports ramp-up of the KL-weight beta via the functionsset_beta!andget_beta.
Network helpers
save_network- save as jld2 filecopy_network- copy from and to GPU
Layers
Fully connected layers
Convolutional
Layers for convolutional networks:
Recurrent
Layers for recurrent networks:
Recurrent- type for recurrent layersRecurrentUnit- type for recurrent units
Helpers for recurrent networks
get_hidden_statesget_cell_statesset_hidden_states!set_cell_states!!reset_hidden_states!reset_cell_states!
Other layers
Attention Mechanisms
Tranformer API
Transformer- generic transformer type, works on tensors of embedded sequences.TokenTransformer- generic transformer type, works on tokenized sequences.TFEncoder- Bert-like transformer encoderTFDecoder- Bert-like transformer decoder
Activation functions
Helferlein-style is to provide all functions (such activation or loss functions) as functions. Therefore any function from any package or any custom function may be provided as actf to the layer constructors.
... see
Knet docufor all activation functions provided by Knet (elu,relu,selu,sigm, ...).Helferlein provides some derived funs, such as
leaky_relu,leaky_tanh,leaky_sigmorswish.
Data provider utilities
DataLoader- type for iterator of minibatchesSequenceData- type for iterator of minibatches of sequences
For tabular data
dataframe_readdataframe_minibatch- turn a dataframe into minibatchesdataframe_splitmk_class_ids
For image data
ImageLoader- turn adirectory structure of image files into minibatches
Image to array tools
ImageNet tools
preproc_imagenet_vggpreproc_imagenet_resnetpreproc_imagenet_resnetv2predict_imagenetget_imagenet_classes
Text data
sequence_minibatch- turn a text corpus into minibatches
Text corpus example data download
Iteration utilities
Training
tb_train!- high-level training utility with tenorboard support and (maybe too) many optional arguments
Evaluation and accuracy
Loss functions
Knet.Ops20.nll- Cross-entropy for classifiers (aka negative log likelihood)Knet.Ops20.bce- binary cross-entropy for binary classifiersfocal_nllfocal_bce- ... see
Knet docufor all loss functions provided by Knet.
Accuracy functions
Knet.Ops20.accuracysquared_error_accabs_error_acchamming_dist- Hamming distance-like accuracypeak_finder_acc- accuracy, suitable for peak detection
Other utils
Utils for array manipulation
crop_arrayblowup_arrayrecycle_arrayde_embed- return argmax for a n-dimensional array
Utils for fixing types in GPU context
Datasets
dataset_mit_nsr- logterm ECGsdataset_mnist- MNISTdataset_iris- Fisher's Iris datasetget_tatoeba_corpus- machine translation text corpidataset_pfam- protein sequences dataset
Pretrained networks
Pretrained network weights, derived from Keras applications.