In the grand tradition of the UNIX Grymoire, Unix for the Beginning Mage, Veghead’s Spell Book, and Ars Technica’s guide to command-line wizardry1… I present to you some shell spells I have found handy on occasion.

I’m not promising that these aren’t awful. I’ll promise some are. I’m sure most can be done more elegantly, but each of these was polished to work well enough for its sometime purpose.

stop relying on epochconverter.com or whatever for epoch time conversion

I’ll admit this one is salient to me principally because of work, but it was so easy to do that I feel silly for having lived without it so long.

In your shell rc (you know, .bashrc or .zshrc or whatnot) or something transcluded by it:

epoch () {
  ruby -rtime -e "puts Time.parse('$*').strftime('%s')"
}

The beauty of this is that Ruby’s Time.parse (Date._parse in a false mustache) is horribly permissive, so you can execute epoch 3 hours ago or epoch February 14th, 2022 or whatever, not just a nicely formatted input.

exit if most recent git commit in repo isn’t more recent than a certain length of time

# 5 * 60 is for five minutes
TIME_THRESHOLD=`echo "$(date +%s) - (5 * 60)" | bc`
LATEST=`git reflog -1 --date=unix | grep -o "[0123456789]\{10\}"`
if [ $LATEST -lt $TIME_THRESHOLD ]; then exit 0; fi

given a list of filenames, filter to those whose most recent relevant git commit was within a certain time threshold

This doesn’t run quick. It could be made much faster if you also filtered on file modification time first, but then you have to commit to some assumptions on whether your files are modified locally or from the git remote first.

# 5 * 60 is for five minutes
TIME_THRESHOLD=`echo "$(date +%s) - (5 * 60)" | bc`
find whatever_dir \ # or ls, anything that lists files
| xargs -I blah bash -c 'echo $(git reflog -1 --date=unix --pretty="format:%gd" blah) blah' \ 
| cut -d'{' -f2- \ # now we're just inelegantly mushing input into shape
| sed 's/\}//' \
| grep "[0123456789]\{10\} .*" \ 
| awk -v tt="$TIME_THRESHOLD" '$1>tt' \ # is it more recent? 
| cut -d' ' -f2 # trim off the time

with a URL as the sole command-line argument, create an HTML redirect page matching its folder structure

#!/bin/bash
URL=$1
TIME=`date +%Y-%m-%dT%H:%M:%S%z`
FMT='<html><head><meta charset="UTF-8"><meta http-equiv="refresh" content="0; URL=%s"/><meta property="article:modified_time" content="%s"/></head><body><p>redirecting you...</p></body></html>'
cd $WHEREVER_YOU_WANT_THIS_STUFF_STORED

# I have more checks exiting out under certain
# conditions because I'm too lazy to filter before
# invoking the script

FNAME=`echo $1 | cut -d'/' -f4-`
if [[ ! $FNAME == *.html ]]
then
	if [[ ! $FNAME == */ ]]
	then
	FNAME="$FNAME/index.html"
	else
	FNAME="$(echo $FNAME)index.html"
	fi
fi
FOLDER=`echo $FNAME | rev | cut -d'/' -f2- | rev`
if [[ ${#FOLDER} -ge 2 ]] # this isn't right but w/e
then
	mkdir -p $FOLDER
fi
printf "$FMT" "$URL" "$TIME" >./$FNAME
cd -

use awk to reorder fields

I keep figuring this out from scratch, so let’s just get it down.

In -v OFS='' the -v signposts there’s going to be an assignment, and OFS stands for Output Field Separator. This is relevant for some kinds of processing that… I don’t do, mostly, because I am always taping things together manually per case.

awk -F'separator' -v OFS='' '{print $2 " " $1}'

how an unethical person might unpackage all those webfonts for local install

If you found cool fonts on a website and then used a handy browser feature to download them, but they were in woff2 format, you might then with this tool:

ls *.woff2 | xargs -I% woff2_decompress %

remove a newline at the end of a file

I don’t want to know why I need to do this sometimes and I wish it didn’t involve perl but let’s be real about using the best tool for the job. Source.

perl -pi -e 'chomp if eof' $YOUR_FILE_SITTING_THERE_BEAUTIFULLY_WITH_A_TERMINAL_NEWLINE

use sqlite to manipulate little data files

From Simon Willison:

sqlite3 :memory: -cmd '.import -csv taxi.csv taxi' \
  'SELECT passenger_count, COUNT(*), AVG(total_amount) FROM taxi GROUP BY passenger_count'

I’m not 100% but I think this might work for JSON:

sqlite3 :memory: -cmd '.mode json' -cmd '.import taxi.json taxi' \
  'SELECT passenger_count, COUNT(*), AVG(total_amount) FROM taxi GROUP BY passenger_count'

But obviously JSON structures aren’t necessarily table-friendly, so I’d bet you’d want to jq the hell out of whatever you were using first.

  1. I am all too happy to expand this list as I find or remember more. ↩