Read EPUB from Terminal 25/12/2018

The REAL Nerd do not use an EBook Reader. Use Terminal Emulators ONLY! :-D

EPUB file are easy to read. Technically an EPUB file is a ZIP file with containing a bunch on HTML files. So, it is pretty simple to read an EPUB using LYNX or W3M (terminal based browsers).

Using LYNX

unzip -l -p MY_EPUB_FILENAME text/* | lynx --stdin

Using W3M

unzip -l -p MY_EPUB_FILENAME text/* | w3m -T text/html

BASH Functions

Save this functions inside your .bashrc or .bash_profile file and you will have a persistent terminal command function to be used to read your lovely loved ebooks:

#read ebook with lynx
function epubl {
    unzip -l -p "$1" text/* | lynx --stdin
}
#read ebook with w3m
function epubw {
    unzip -l -p "$1" text/* | w3m -T text/html
}

Have FUN!

Piano from Terminal 02/09/2018

I'm learning to play Piano but my terminal call me.. What about a simple piano visualizer in the terminal?

I need a simple visualizer that highlights piano keys.. I don't need it, really, but its fun to make everythings in the terminal (or not?). This visualizer looks like this:

Coding

I have tried to write this tool with a "simple" bash script but i know C and i am more "productive" with C. So this is the main structure of the program:

/*p[Y][NOTE]
    where p[0] is the top line of the piano
          p[1] is the middle line of the piano
          p[2] is the bottom line of the piano
    foreach p[COLUMN] there are 14 keys for each OCTAVE
        [0] is the C key
        [1] is the C# key
        [2] is the D key
        [3] is the D# key
        [4] is the E key
        [5] is a dummy key for a not existent semitone
        [6] is the F key
        ...
        [12] is the A# key
        [13] is a dummy key for a not existent semitone 
        */
char p[3][14*N_OCT]={0};

//[...]
int main(int argc, char *argv[]) {
    //check argv for arguments

    //for each note in arguments hightlight p[0..2][NOTE]

    //draw all piano keys
}

The trick of using 14 keys instead of the standard 12 is used to calculate a key index, trought octaves, in a simpler way, assuming that each tone has a semitone. This is not true for E-F and B-C intervals but the program will intercept this exceptions and skip the [5] and the [13] dummy keys (not existent semitones).

This is how this trick work:

1) Notes are translated to numbers:

Notes: CDEFGAB
NOTE_INDEX: C=0 D=2 E=4 F=6 G=8 A=10 B=12

2) This formula is used to find the index of a key for a note without alterations:

KEY_INDEX = NOTE_INDEX(NOTE)*2 + (14 * OCTAVE)

3) (optional) code to add a semitone to a KEY_INDEX:

ALTERATED_KEY_INDEX: KEY_INDEX +1
IF (ALTERATED_KEY_INDEX==5 OR 13)
    ALTERATED_KEY_INDEX = ALTERATED_KEY_INDEX +1;

4) (optional) code to add two semitones (##):

two time the first algorithm

5) (optional) code to subtract a semitone (b):

one time the first algorithm with -1 instead of +1

6) (optional) code to subtract two semitones (bb):

two time the subtract of a single semitone

By using The trick of 14 keys, instead of 12, the program is pretty simple. For each note (ex: A2#) passed as an argument, the program will highlight the key using Terminal colors. After highlights of all keys the program draw all 12 keys (skipping the 5 and the 13 dummy keys).

Sourcecode

The full source of this program is here. The program use ANSI ESCAPE CODE to colorize the terminal output, so will run fine on a Linux machine, on Mac OS X or with MinGW on Windows.

Compile

To compile this program you do not need any fancy arguments:

gcc piano.c -o piano

Usage

This program accept a list of notes where each note syntax is

NOTE[OCTAVE][ALTERATION][ALTERATION]

For example if you want to highlight a C# you write C# (or B), if you want a C# on the second octave you write C2# (or B2), if you want a C with two semitones alteration you write C## (or B#).

By default the program will display 3 octaves but you can change that by using the -o (or --octaves) parameters.

The maximum number of octaves supported by the program is 10. You can extend this values by modifing the sourcecode, changing the value of the N_OCT macro.

Ok, this tool is useless... but exists :-D

YouTube from Terminal (Part 2) 02/09/2018

This is the second part of the Nerd guide to play Youtube videos from your terminal.

In the first part these bash function has been added to my .bashrc (or .bash_profile on Mac OS X):

function yt {
    mpv --ontop --geometry=320x200+0+0 $(youtube-dl -g -f 18 $1)    
}
#download Youtube video using youtube-dl
function dyt {
    curl $(youtube-dl -g -f 18 $1) -o video.mp4
}

Now, the missing part is to search something and play all results. The follow solution will play all results in sequence. Youtube results are paginated on multiple HTML pages. In this case this script will play only the first page of results (who care.. these NERD scripts are insane):

#search Youtube and play all results
function yts {
    curl "https://www.youtube.com/results?search_query=$(echo $* | sed -E 's/ /%20/g')" -L | grep -o '/watch?[^\"]*' | uniq | grep -v "list=" | while read -r line; do yt "https://www.youtube.com$line"; done;
}

To use this function you only need only to type

yts MY LONG SEARCH STRING

To skip a video that you do not like you need to close mpv window. To terminate the script (looping throught all results) you need to send CTRL-C to the terminal running yts.

If you want to download all files, instead of playing them, you can use dyt instead of yt.

Let's break this function in pieces:

$(echo '$*' | sed -E 's/ /%20/g')

Grab all parameters passed to yts function ($*) and replace all whitespaces with %20 (the space character encoded for HTML URL)

curl "https://www.youtube.com/results?search_query=$(echo '$*' | sed -E 's/ /%20/g')" -L

Download youtube search results, following HTTP redirection (-L)

grep -o '/watch?[^\"]*' | uniq

Extract all /watch?* video URL. Some URL are duplicated so with uniq the output of the command is a list of unique URLs without duplicates

grep -v "list="

Exclude all lines that contains "list="; these lines are for Youtube playlists

while read -r line; do yt "https://www.youtube.com$line"; done;

Loop throught each line. For each line execute yt function to view the video (or dyt if you want to download videos).

Full code for play and search

#play Youtube video using youtube-dl
function yt {
    mpv --ontop --geometry=320x200+0+0 $(youtube-dl -g -f 18 $1)    
}
#download Youtube video using youtube-dl
function dyt {
    curl $(youtube-dl -g -f 18 $1) -o video.mp4
}
#search Youtube and play all results
function yts {
    curl "https://www.youtube.com/results?search_query=$(echo '$*' | sed -E 's/ /%20/g')" -L | grep -o '/watch?[^\"]*' | uniq | grep -v "list=" | while read -r line; do dyt "https://www.youtube.com$line"; done;
}

YouTube from Terminal (Part 1) 02/09/2018

Sometimes a NERD watch Youtube without a browser opened. NERDS use text browsers like LYNX or W3M :-D

In the past years i've tryed mps-youtube and youtube-viewer, both are terminal apps that allow you to search, view and download videos from Youtube, but this time i want something more customizable, also based on youtube-dl.

Solution 1: play video from URL

You need only these two tools: youtube-dl and mpv. I think that these are the same tools used by mps-youtube. This solution is quick and dirty.

Installation on Mac (via brew) is easy:

brew install youtube-dl mpv

On Linux, Debian based distros, you can use apt:

apt-get install youtube-dl mpv

After that you can open your video with this command (replace VIDEO_URL with your Youtube video URL):

mpv $(youtube-dl -g -f 18 VIDEO_URL)

The magic number 18 is to select MP4 360p videos. If you want to select others format you can choose from one of these:

MAGIC NUMBER	FORMAT	RESOLUTION	CODECS
171	webm	audio only	vorbis
140	m4a	audio only	mp4a
133	mp4	426x240	avc1 video only
134	mp4	640x360	avc1 video only
135	mp4	854x480	avc1 video only
247	webm	1280x720	vp9 video only
136	mp4	1280x720	avc1 video only
248	webm	1920x1080	vp9 video only
137	mp4	1920x1080	avc1 video only
43	webm	640x360	vp8 + vorbis
18	mp4	640x360	avc1 + mp4a
22	mp4	1280x720	avc1 + mp4a

If you want to be lazy in typing commands (like a real NERD needs to be), you can add this function to your .bashrc (or .bash_profile if you use Mac OS X) and use have this solution ready anytime you need to watch videos from terminal:

function yt {
    mpv $(youtube-dl -g -f 18 $1)   
} 

To use this function you only need only to type

yt VIDEO_URL

Download videos

If you want to download videos, instead of playing them, you only need to replace mpv with curl:

function dyt {
    curl $(youtube-dl -g -f 18 $1) -o video.mp4
} 

Picture in Picture alike

When i work on my computer i like to have the video that i am watching on one corner of the screen, possibly on top of the others windows (this mode is normally called PIP: picture in picture). With mpv you can use two parameters to configure the position of the window and the stay-on-top mode:

mpv --ontop --geometry=WIDTHxHEIGHT+X+Y

Add these parameters to your scripts to have mpv play like a PIP!

Experiment without youtube-dl (little more complex)

You don't like to use existent tools. You are a NERD and you like to do everything with a bunch of curl, grep and sed commands. Supposing that you do not wont to rewrite curl or mpv (you dont?), you can use curl to extract the required video URL and play that with mpv:

function ytc {
    mpv $(echo -ne $(curl $1 | grep -o 'https[^\"]*' | grep videoplayback | grep itag%3D18 | sed -E "s/%/\\\\x/g"))
}

This is it. HTML scraping: the worst thing you should do, EVER. Let's break each command apart:

curl $1 | grep -o 'https[^\"]*'

curl youtube page and extract all HTTPS links contained in the page. If you look at the HTML page returned from Youtube, you will see that inside HTML there is a big JSON string that contains all HTTPS URLs for the final video playback. These links contains the videoplayback substring. see screenshot-1

grep videoplayback

select only lines that contains videoplayback substring

grep itag%3D18

select only lines with itag=18 (my preferred format is 18, you can choose others format from the table above). %3D is the character = escaped inside the HTTP URL

sed -E "s/%/\\\\x/g"

convert each HTTP encoded characters from %NN to \xNN. This is the standard C/bash escape for characters expressed as hexadecimal two digit number

echo -ne

echo results converting all \xNN sequences to its ASCII equivalents. After that you obtain the final URL. Something like this: r11---sn-hpa7zn7r.googlevideo.com/videoplayback?initcwndbps=472500&sparams=clen%2Cdur%2Cei%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cnh%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&ip=87.8.231.48&key=yt6&mn=sn-hpa7zn7r&signature=23D980F8415F6F313EF422E95BA85AA55FCD22BA.96645A049DC436870F36DD48BFE64DE20283F787&mm=31&ms=au&id=o-AP4my72PmmcNCaYptxcQ92Q3knmzotCfM9FaAHfHAW5E&ei=PAw6WP2GE4HXcPy_haAF&mv=m&mt=1480198892&dur=590.483&lmt=1428034888381069&source=youtube&upn=vncEZvHsRZc&itag=18&requiressl=yes&clen=40049035&nh=IgpwcjA0Lm1pbDAxKgw3Mi4xNC4yMDQuNzM&ipbits=0&ratebypass=yes&pl=19&expire=1480220828&gir=yes&mime=video%2Fmp4

mpv

run mpv, finally

PLEASE NOTE that this solution do not work for all Youtube videos. Use youtube-dl instead, it works better and it is less insane.

Full code

#play Youtube video using youtube-dl
function yt {
    mpv --ontop --geometry=320x200+0+0 $(youtube-dl -g -f 18 $1)    
}
#download Youtube video using youtube-dl
function dyt {
    curl $(youtube-dl -g -f 18 $1) -o video.mp4
}
#play Youtube video without youtube-dl (do not work for all videos) 
function ytc {
    mpv $(echo -ne $(curl $1 | grep -o 'https[^\"]*' | grep videoplayback | grep itag%3D18 | sed -E "s/%/\\\\x/g"))
}
#download Youtube video without youtube-dl (do not work for all videos) 
function ytc {
    curl $(echo -ne $(curl $1 | grep -o 'https[^\"]*' | grep videoplayback | grep itag%3D18 | sed -E "s/%/\\\\x/g")) -o video.mp4
}

Have FUN (or use a normal, more easy, soft clickable, quite adorable WebBrowser like everyone do)