Publications

2022

July

AfriTeVa: Extending “Small Data” Pretraining Approaches to Sequence-to-Sequence Models

Ogundepo et al.

Lightweight Transformers for Conversational AI

Pressel et al.

SSBNet: Improving Visual Recognition Efficiency by Adaptive Sampling

Kwan & Song

Detecting and mitigating issues in image-based COVID-19 diagnosis

Silva, Rezende & Ponti

Recurrent Connections in the Primate Ventral Visual Stream Mediate a Trade-Off Between Task Performance and Network Size During Core Object Recognition

Nayebi et al.

StyleGAN2 Flax TPU

Müller & Laurent

Yes, No or IDK: The Challenge of Unanswerable Yes/No Questions

Sulem, Hay & Roth

Language Modelling with Pixels

Rust et al.

Use of Deep Learning to Detect the Maternal Heart Rate and False Signals on Fetal Heart Rate Recordings

Boudet et al.

June

Improving Ensemble Distillation With Weight Averaging and Diversifying Perturbation

Nam et al.

Evaluating Generative Patent Language Models

Lee

Thermalization and localization in isolated many-body quantum systems

Morningstar

Unified-IO

Lu et al.

Pre-training and Evaluating Transformer-based Language Models for Icelandic

Daðason & Loftsson

A Dense Representation Framework for Lexical and Semantic Matching

Lin & Lin

Insights into Pre-training via Simpler Synthetic Tasks

Wu, Li & Liang

Cachew: Machine Learning Input Data Processing as a Service

Graur et al.

Visualizing attention zones in machine reading comprehension models

Cui, Zhang & Liu

Edge Inference with Fully Differentiable Quantized Mixed Precision Neural Networks

Schaefer et al.

Xplique: A Deep Learning Explainability Toolbox

Fel et al.

DALL·E mini

Dayma

Can CNNs Be More Robust Than Transformers?

Wang et al.

Turtle Recall

Ivanov

Channelized Axial Attention - Considering Channel Relation within Spatial Attention for Semantic Segmentation

Huang et al.

Squeezeformer: An Efficient Transformer for Automatic Speech Recognition

Kim et al.

May

Describing Differences between Text Distributions with Natural Language

Zhong et al.

Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval

Notin et al.

t5_base_NCC

Kummervold

nafnet-jax

Li

EBM Life Cycle: MCMC Strategies for Synthesis, Defense, and Density Modeling

Hill et al.

hmBERT: Historical Multilingual Language Models for Named Entity Recognition

Schweter et al.

Multimodal Masked Autoencoders Learn Transferable Representations

Geng et al.

Zero-Shot and Few-Shot Learning for Lung Cancer Multi-Label Classification using Vision Transformer

Guo & Fan

Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval

Gao & Callan

Inception Transformer

Si et al.

Semi-self-supervised Automated ICD Coding

Hlynsson et al.

xcit

Dagli

Generating Disentangled Arguments With Prompts: A Simple Event Extraction Framework That Works

Si et al.

Multilingual multi-aspect explainability analyses on machine reading comprehension models

Cui et al.

ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language

Phan et al.

Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning

Liu et al.

Long Document Re-ranking with Modular Re-ranker

Gao & Callan

Experiments with big_vision

Paul

On the Design of 2D Human Pose Estimation Networks using Accelerated Neuroevolution and Novel Keypoint Representations

McNally

April

Hierarchical Label-wise Attention Transformer Model for Explainable ICD Coding

Liu et al.

Cross-stitched Multi-modal Encoders

Singla et al.

Multi-label topic classification for COVID-19 literature with Bioformer

Fang & Wang

Density Matrix Renormalization Group with Tensor Processing Units

Ganahl et al.

What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?

Wang et al.

Scalable Semi-Modular Inference with Variational Meta-Posteriors

Carmona & Nicholls

March

Pytorch Hyperparameter Optimization on TPUs

Suess

Code Clippy Github Dataset

Code.AI

BioBERTurk- Turkish Biomedical Language Models

Türkmen

Towards Efficient and Scalable Sharpness-Aware Minimization

Liu et al.

Plumber: Diagnosing and Removing Performance Bottlenecks in Machine Learning Data Pipelines

Kuchnik et al.

Skillful Precipitation Nowcasting — An Implementation of DeepMind's DGMR

Hassaan

Learning neural audio features without supervision

Yadav & Zeghidour

STaR: Bootstrapping Reasoning With Reasoning

Zelikman, Wu & Goodman

PACS: A Dataset for Physical Audiovisual CommonSense Reasoning

Yu et al.

TPU VM trained weight release w/ PyTorch XLA

Wightman

Dynamics of Transmon Ionization

Shillito et al.

KinyaBERT: a Morphology-aware Kinyarwanda Language Model

Nzeyimana & Rubungo

PERT: Pre-training BERT with permuted language model

Cui, Yang & Liu

Tevatron: An Efficient and Flexible Toolkit for Dense Retrieval

Gao et al.

Active Evaluation: Efficient NLG Evaluation with Few Pairwise Comparisons

Mohankumar & Khapra

IT5: Large-scale Text-to-text Pretraining for Italian Language Understanding and Generation

Sarti & Nissim

Adversarial Robustness of Neural-Statistical Features in Detection of Generative Transformers

Crothers et al.

The evolution, evolvability and engineering of gene regulatory DNA

Vaishnav et al.

February

Development of a Construction Specialized Pretrained Language Model

Kim

Slovak GPT-J-1.4B

Kondela

Recurrent Connections in the Primate Ventral Visual Stream Mediate a Tradeoff Between Task Performance and Network Size During Core Object Recognition

Nayebi et al.

TPU Starter

Mikazuki

audax

Yadav

CEDILLE: A large autoregressive language model in French

Müller & Laurent

Tensor Processing Units as Quantum Chemistry Supercomputers

Pederson et al.

Environment Classification for Robotic Leg Prostheses and Exoskeletons Using Deep Convolutional Neural Networks

Laschowski et al.

Icelandic ConvBERT-Base

Daðason

JaQuAD: Japanese Question Answering Dataset for Machine Reading Comprehension

So et al.

January

Nordic ELECTRA-Small

Daðason

Icelandic-Norwegian ELECTRA-Small

Daðason

Our Summer of Code Project on TF-GAN

P A, Maynard-Reid & Shor

BioM-Transformers: Building Large Biomedical Language Models with BERT, ALBERT and ELECTRA

Alrowili & Vijay-Shanker

Interactive Gated Decoder for Machine Reading Comprehension

Cui et al.

Making and Using AI in the Library: Creating a BERT Model at the National Library of Sweden

Haffenden, Fano & Malmsten

A Large and Diverse Arabic Corpus for Language Modeling

Ali

MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound

Zellers et al.

GPT Neo 1.3B pre-trained on cleaned Dutch mC4 🇳🇱

Havinga

GPT-Neo 125M pre-trained on cleaned Dutch mC4 🇳🇱

Havinga

GPT2-Large pre-trained on cleaned Dutch mC4 🇳🇱

Havinga

Distributed Evolution Strategies Using TPUs for Meta-Learning

Sheng & He

Icelandic ELECTRA-Base

Daðason

Icelandic ELECTRA-Small

Daðason

2021

December

BeerAI

Beckmann

Information retrieval and question answering: A case study on COVID-19 scientific literature

Otegi et al.

How and What to Learn: Taxonomizing Self-Supervised Learning for 3D Action Recognition

Tanfous et al.

BiCSNet: A Bidirectional Cross-Scale Backbone for Recognition and Localization

Peng et al.

Learned Queries for Efficient Local Attention

Arar, Shamir & Bermano

CPPE-5: Medical Personal Protective Equipment Dataset

Dagli & Shaikh

Contrastive Self-Supervised Pretraining of Vison Transformers

Cascio

GPT2-Medium pre-trained on cleaned Dutch mC4 🇳🇱

Havinga

T5-large pre-trained on cleaned Dutch mC4 🇳🇱

Havinga

Vision Transformer(ViT) in Tensorflow2

Lee

Large Scale Distributed Linear Algebra With Tensor Processing Units

Lewis et al.

Mask-combine Decoding and Classification Approach for Punctuation Prediction with real-time Inference Constraints

Minixhofer, Klejch & Bell

WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models

Minixhofer, Paischer & Rekabsaz

Simple Allocation Rules And Optimal Portfolio Choice Over The Lifecycle

Duarte

S3NAS: Fast Hardware-aware Neural Architecture Search Methodology

Lee et al.

Densifying Sparse Representations for Passage Retrieval by Representational Slicing

Lin & Lin

A Novel Method for Automated Identification and Prediction of Invasive Species Using Deep Learning

Elias

November

Interpreting intermediate feature representations of raw-waveform deep CNNs by sonification

Yadav

LASSL

Kim et al.

Chase World

Kassa

leaf-pytorch

Yadav

RoGPT2: Romanian GPT2 for Text Generation

Niculescu, Ruseti & Dascalu

ArabicTransformer: Efficient Large Arabic Language Model with Funnel Transformer and ELECTRA Objective

Alrowili & Vijay-Shanker

SkipResNet

Takeda

Poolformer

Sea AI Lab

Simulation of quantum physics with Tensor Processing Units: brute-force computation of ground states and time evolution

Hauru et al.

MetaFormer is Actually What You Need for Vision

Yu et al.

Building Keras from Source: A Follow-Along Guide

Kane

RoBERTuito: a pre-trained language model for social media text in Spanish

Pérez et al.

Historic Language Models (HLMs)

Schweter

Simulation of quantum many-body dynamics with Tensor Processing Units: Floquet prethermalization

Morningstar et al.

Solving Inverse Problems in Medical Imaging with Score-Based Generative Models

Song et al.

October

StaResGRU-CNN with CMedLMs:A stacked residual GRU-CNN with pre-trained biomedical language models for predictive intelligence

Ni et al.

DarijaBERT

Gaanoun

AraBERTv0.2-Twitter

AUB Mind Lab

SIMONe

Le-Khac

DARLING: Deep leARning for chemicaL InformationprocessinG

Rajan

Do We Know What We Don't Know? Studying Unanswerable Questions beyond SQuAD 2.0

Sulem, Hay & Roth

Post-correction of OCR Results Using Pre-trained Language Model

Piotrowski

Search-Augmented Question Answering System Using Multilingual Transformer Model

Piotrowski

MutFormer: A context-dependent transformer-based model to predict pathogenic missense mutations

Jiang, Fang & Wang

Redesigning the Transformer Architecture with Insights from Multi-particle Dynamical Systems

Datta et al.

CrabNet for explainable deep learning in materials science: bridging the gap between academia and industry

Wang et al.

Multitask Prompted Training Enables Zero-Shot Task Generalization

Sanh et al.

SELECTRA: A Spanish ELECTRA

Fidalgo et al.

AIのべりすと

Kousin

Delphi: Towards Machine Ethics and Norms

Jiang et al.

The Inductive Bias of In-Context Learning: Rethinking Pretraining Example Design

Levine et al.

Cut the CARP: Fishing for zero-shot story evaluation

Matiana et al.

ResNet strikes back: An improved training procedure in timm

Wightman, Touvron & Jégou

September

Icelandic ConvBERT-Small

Daðason

Training on Test Data with Bayesian Adaptation for Covariate Shift

Zhou & Levine

O-JMeSH: creating a bilingual English-Japanese controlled vocabulary of MeSH UIDs through machine translation and mutual information

Soares et al.

JAX vs PyTorch: A simple transformer benchmark

Nolan

The Challenge of Appearance-Free Object Tracking with Feedforward Neural Networks

Malik et al.

AraT5: Text-to-Text Transformers for Arabic Language Understanding and Generation

Nagoudi, Elmadany & Abdul-Mageed

Clustering Monolingual Vocabularies to Improve Cross-Lingual Generalization

Bassani

Pretrained Neural Models for Turkish Text Classification

Okur & Sertbaş

Augmenting BERT-style Models with Predictive Coding to Improve Discourse-level Representations

Araujo et al.

BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translation

Xu, Van Durme & Murray

ReasonBERT: Pre-trained to Reason with Distant Supervision

Deng et al.

gpt-c

Grankin

ChessCoach

Butner

Performance of chemical structure string representations for chemical image recognition using transformers

Rajan, Zielesny & Steinbeck

TUNiB-Electra

Kim et al.

An Approach to Extractive Bangla Question Answering Based On BERT-Bangla And BQuAD

Saha et al.

TRC로 월 몇만원에 GPU 수십개급의.. TPU 사용 가능

Lee

Characterizing Possible Failure Modes in Physics-Informed Neural Networks

Krishnapriyan et al.

An Empirical Exploration in Quality Filtering of Text Data

Gao

May

ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning

Yao et al.

Wav2Vec2

Gupta

Flexible Architectures for Image Synthesis

Jain

EfficientNet JAX - Flax Linen and Objax

Wightman

TensorFlow Datasets IO (tfdsio)

Nguyen

Scientific Claim Verification with VERT5ERINI

Pradeep et al.

Detecting Anatomical and Functional Connectivity Relations in Biomedical Literature via Language Representation Models

Ozyurt et al.

BioELECTRA:Pretrained Biomedical text Encoder using Discriminators

Kanakarajan, Kundumani & Sankarasubbu

BioM-Transformers: Building Large Biomedical Language Models with BERT, ALBERT and ELECTRA

Alrowili & Vijay-Shanker

Stress Test Evaluation of Biomedical Word Embeddings

Araujo et al.

Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Level

Zhong et al.

How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering

Jiang et al.

CoolMomentum: a method for stochastic optimization by Langevin dynamics with simulated annealing

Borysenko & Byshkin

Tensorflow2 기반 Seq2Seq 모델, 학습, 서빙 코드 구현

Park

DeepDarts: Modeling Keypoints as Objects for Automatic Scorekeeping in Darts using a Single Camera

McNally et al.

KLUE: Korean Language Understanding Evaluation

Park et al.

How Deep is your Learning: the DL-HARD Annotated Deep Learning Dataset

Mackie, Dalton & Yates

hebrew-gpt_neo

Adler

Unbiased Monte Carlo Cluster Updates with Autoregressive Neural Networks

Wu et al.

April

Contextualized Query Embeddings for Conversational Search

Lin, Yang & Lin

Mesh Transformer JAX

Wang

DECIMER1.0: Deep Learning for Chemical Image Recognition using Transformers

Rajan, Zielesny & Steinbeck

Clinical BERT Models Trained on Pseudo Re-identified MIMIC-III Notes

Lehman et al.

Operationalizing a National Digital Library: The Case for a Norwegian Transformer Model

Kummervold et al.

Categorising Vaccine Confidence with TransformerBased Machine Learning Model: The Nuances of Vaccine Sentiment on Twitter

Kummervold et al.

City-Scale Simulation Of Covid-19 Pandemic & Intervention Policies Using Agent-Based Modelling

Suryawanshi et al.

CodeTrans: Towards Cracking the Language of Silicone's Code Through Self-Supervised Deep Learning and High Performance Computing

Elnaggar et al.

Arabic Compact Language Modelling for Resource Limited Devices

Alyafeai & Ahmad

Igor Ivanov: Harnessing Machine Learning Skills to Reduce Damages from Tropical Storms

Radiant Earth Foundation

Computer Vision and Deep Learning for Environment-Adaptive Control of Robotic Lower-Limb Exoskeletons

Laschowski et al.

InAugment: Improving Classifiers via Internal Augmentation

Arar, Shamir & Bermano

IndT5: A Text-to-Text Transformer for 10 Indigenous Languages

Nagoudi et al.

Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages

Ramesh et al.

Self-Supervised Representation Learning with Relative Predictive Coding

Tsai et al.

Virtual Sensing and Sensors Selection for Efficient Temperature Monitoring in Indoor Environments

Brunello et al.

2020

2019

February

Diagnose and Explain

d'Almeida

2018

Don't see your TRC-supported work here?

Please let us know about it by filling out this short form .