Blog – Arumoy Shome

This page contains links to some of my writings on topics that interest me. Usually, they are inspired by problems that I experience in my day-to-day life.

I like pondering over the act or the process of doing something.

Machine Learning Fairness Testing Should be More Holistic

research

machine learning

fairness

testing

SE4AI

data centric

Extracting the Module and Function Names from Python ASTs

python

How to extract the module and function name from Python Abstract Syntax Trees.

Visualisation Zoo

python

design

Collection of data visualisations I have created using Python.

Bash commands

shell

Bash (bash) commands I frequently use.

Docker commands

shell

Docker (docker) commands I frequently use.

Organising research projects with git

research

productivity

Some standards and conventions I follow when organising research project data using git.

Toggling background color in kitty and vim (yob)

shell

vim

yob is a tiny shell script which toggles between a light and dark colorscheme in Kitty, my terminal of choice.

Managing Scientific Bibliography using Emacs Org-mode

emacs

productivity

How I organise, search and retrieve my scientific papers using Emacs org-mode.

Automatically retrieving Bibtex information from DOI

shell

productivity

doi2bib is a simple Python script I wrote that fetches bibtex information from the Crossref API using the provided DOI. It can also handle pre-prints published on Arxiv.

Aspell Commands

shell

Aspell (aspell) commands I frequently use.

Navigating large markdown files in vim

shell

vim

Strategies to navigate large plaintext (specifically markdown) files in vim.

CMS using Pandoc and Friends

shell

web

Some tools & techniques I use to run a no non-sense blog using static html pages. All powered by a sane file naming convension, plaintext documents writing in markdown and exported to html using pandoc and other unix cli tools.

Packing and Unpacking `**kwargs`

python

Packing and unpacking keyword arguments in Python.

Ripgrep commands

shell

Ripgrep (rg) commands I frequently use.

Data Validation with TFDV

data

research

SE4AI

In this lecture we will go over the basics of data validation. The first half of this lecture will be a talk on the fundamentals of data validation. We will answer what is data validation?, why should we validate our data? and how we can validate our data?. The second half of the lecture will be a hands-on tutorial on using Tensorflow Data Validation, instructions & code for which can be found on this github repo.

Effortless Parallel Execution with xargs & Friends

shell

Recently, I had to run Tensorflow Data Validation on over 500 public datasets from Kaggle to generate a baseline schema file for further analysis. I chose to do this using the xargs unix command.

Data Smells in Public Datasets

data

research

SE4AI

technical debt

In this talk I will present our recent paper titled Data Smells in Public Datasets which was published at the 1^st International Conference on AI Engineering (CAIN) 2022. I will first present the problem we are trying to solve along with the contributions that we made. I will present the methodology which was followed along with the results obtained. I will present a select few smells which I personally find interesting & hope will generate some discussion. Finally, we will conclude the talk with some high level takeaways from our study along with the limitations & future directions of work.

There and Back Again A Tale of Website Management

shell

vim

web

Managing websites using markdown, shell and vim.

Timestamps in the Shell (today)

shell

productivity

Creating timestamps in the terminal.

Aru’s Information Management System (AIMS)

shell

productivity

AIMS or Aru’s Information Management System is a collection of shellscripts to manage information in plaintext. It is inspired by org-mode, and tries to replicate a subset of its functionalities which I frequently use. AIMS is completely tuned towards my workflow as a researcher and how I manage my digital notes.

Reflections on Scientific Research

research

Reflections on the scientific process and what constitutes being a good researcher.

Privacy Preserving Deep Learning

machine learning

research

privacy

A talk on Privacy Preserving Deep Learning (PPDL) I gave to my research group. It was largly based on a literature review I did during my Msc.

Focus

productivity

Notes on attaining focus.

Project Management

productivity

Notes on project management.

Research Workflow in Plaintext

productivity

emacs

research

In this talk I will go over how we can use Emacs and org-mode to craft a research workflow. We will look at how we can leverage the power of Emacs and org-mode to capture, store, search and retrieve research data, all in plain text! The talk will touch upon how org-mode can be used as an environment for literate programming and reproducible research. I do not assume any prior knowledge of emacs or org-mode and I want this to be more of a discussion rather than a talk. Please ask me questions as I go along and share your thoughts, tips and techniques with others!

Nutrition

misc

Notes on diet and nutrition.

Aru’s Org Capture Template (aocp.el)

emacs

productivity

An Emacs package I wrote for managing bibliographic information.

Scientific Paper Discovery

research

My process of discovering relevant and important papers in a new scientific field.

Research Workflow

productivity

research

An outline of my research workflow which I have developed to handle the non-linear nature of scientific work.

Productivity

productivity

Productivity to me comes from proper information management. Information can come in many forms: email, thoughts/ideas, deadlines, events, project planning, online links, and many more.

Dired Commands

emacs

Dired commands I frequently use.

Garbage Collection for Matplotlib in org-mode

python

emacs

Notes on managing memory when using org-mode and python to conduct data science analysis.

Python context file

python

Python context file for managing imports.

shell `path_finder`

shell

How $PATH is constructed on OSX.

Navigating in Vim

vim

Strategies to navigate files in vim.

Searching in Vim

vim

Various strategies for searching text within vim.

Private Link Sharing

web

Poor man’s document sharing with private links, sort of what you get with Google Docs.