Data Science at the Command Line

This thoroughly revised guide demonstrates how the flexibility of the command line can help you become a more efficient and productive data scientist. You’ll learn how to combine small yet powerful command-line tools to quickly obtain, scrub, explore, and model your data. To get you started, author Jeroen Janssens provides a Docker image packed with over 100 Unix power tools.
How to listen to music from the console using the cmus player on Linux
We can cycle through the reproduction modes by pressing the m
key. The modes define the range of song which are considered in the library view. When using the all from library mode, all the song in the library will be considered. In the artist from library mode the player will consider only the songs in the library which belongs to the same artist of the song which is currently playing. Finally, in the album from library mode, the player will only consider the songs belonging to the same album of the one currently in reproduction.
If the C
function is active, as in our case, when cmus finishes reproducing the currently playing song, it will automatically start playing the next one in the list. To toggle this functionality we must press shift+c
.
If the R
function is active cmus will start again reproducing the group of songs we selected once it reaches the end of the list. For example, if we are using the artist from library mode, once all the songs of the artist are played, the player will start reproducing the artist’s collection from the start. For this function to be effective the C
function must also be active. When the S
function is active, the player is in shuffle mode, so it will choose random tracks from the library.
When the F
function is active the player will always automatically select the current playing song on the list. To be more clear, suppose we have the C
function enabled: when the player finishes playing the currently selected song it will start playing the next in the list, but, by default, it will not “select” it (it will not highlight it with the “selection bar”). When the F
function is enabled, instead, the two things will always be in sync, and the song currently in reproduction will always be selected automatically.
How To Find My Public IP Address From Linux CLI
Type the following dig (domain information groper) command on a Linux, OS X, or Unix-like operating systems to see your own public IP address assigned by the ISP:
dig +short myip.opendns.com @resolver1.opendns.com
You can also type:
dig TXT +short o-o.myaddr.l.google.com @ns1.google.com
Converting my OTF font into multiple web fonts with this bash script
I came across this use-case where we had to use a specific custom font but it was only available in .otf
. However, we want to support multiple formats to ensure even deprecated browsers can load the font. Otf has a global coverage of 97.89% but we didn’t want to take any chances of the font not loading as it’s a crucial feature in our app.
I wanted to convert the font to support the following browsers:
- woff2 – Latest browsers
- woff – Modern browsers
- ttf – Apple and mobile OS
svg– older safari and ios support- eot – older IE support
Using ffmpeg to compress, convert, and resize videos
To compress for web at a reasonable broadband bitrate of about 1.5Mbps video / 128kbps audio:
ffmpeg -i source.mp4 -c:v libx264 -b:v 1.5M -c:a aac -b:a 128k target.mp4
To scale down a high-resolution source video to something more reasonable for Web (qHD for cellular, HD for broadband), the -filter:v
argument is used:
ffmpeg -i source.mp4 -c:v libvpx-vp9 -b:v 0.33M -c:a libopus -b:a 96k \
-filter:v scale=960x540 target.webm
Ricing your system
This is a book about understanding the parts of your system, and customize them as you please.
You won’t find anything related to GUIs (Graphical User Interfaces) in here. That’s primarily because GUIs aren’t standardized or «equal» on all unix machines. However, terminals are and for this fair reason I’m only going to treat CLIs (Command Line Interface) and TUI (Terminal User Interface.)
Música desde la línea de comandos
…cómo bajarlo flac, un formato de audio de alta calidad [de youtube]:
youtube-dl –extract-audio –audio-format flac https://youtu.be/EIoIM_r2cpc
Para bajar un vídeo con la mejor calidad, primero vemos qué opciones tiene con -F.
youtube-dl -F https://www.youtube.com/watch?v=7s8cQ2cOnl8
Normalmente la de mejor calidad es la 22, por lo que lo especificamos con -f 22:
youtube-dl -f 22 https://youtu.be/EIoIM_r2cpc
Best Tools For Taking and Editing Screenshots in Linux
Deepin Scrot is a slightly advanced terminal-based screenshot tool. Similar to the others, you should already have it installed. If not, get it installed through the terminal by typing:
sudo apt-get install scrot
After having it installed, follow the instructions below to take a screenshot:
To take a screenshot of the entire screen:
scrot myimage.png
To take a screenshot of the selected aread:
scrot -s myimage.png
Linux Bash Scripting Part3 – Parameters and Options
To pass data to your shell script, you should use command line parameters.
Options are single letters with a dash before it.
How to recursively chmod all directories except files?
To recursively give directories read&execute privileges:
find /path/to/base/dir -type d -exec chmod 755 {} +
To recursively give files read privileges:
find /path/to/base/dir -type f -exec chmod 644 {} +
How to Make Simple Graphical Shell Scripts with Zenity on Linux
Zenity adds graphical interfaces to shell scripts with a single command.
Compress PDF files on Linux
Ghostscript is a great tool for compressing PDFs. In our tests it delivered the best compression ratio. Simply run:
gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dPDFSETTINGS=/screen -dNOPAUSE -dQUIET -dBATCH -sOutputFile=out.pdf in.pdf
-dPDFSETTINGS can be any of:
/screen : low-resolution output, lowest output size
/ebook : medium-resolution output, medium output size
/printer OR /prepress : high-resolution with maximum output size
Run the Shrinkpdf script:
./shrinkpdf in.pdf out.pdf
The script is almost same as the explicit gs command above. However, it has one advantage – it does an output file size check.
Solarized. Precision colors for machines and people
Solarized is a sixteen color palette (eight monotones, eight accent colors) designed for use with terminal and gui applications. It has several unique properties. I designed this colorscheme with both precise CIELAB lightness relationships and a refined set of hues based on fixed color wheel relationships. It has been tested extensively in real world use on color calibrated displays (as well as uncalibrated/intentionally miscalibrated displays) and in a variety of lighting conditions.
Listen to the radio at the Linux terminal
MPlayer is an extremely versatile open source media player that can be surprisingly useful at the Linux command line.
MPlayer has a slew of command-line options to set depending on your situation. I wanted to listen to the local college radio station here in Raleigh (88.1 WKNC, they’re pretty good!), and so after grabbing the streaming URL from their website, all that took to get my radio up and running, no GUI or web player needed, was:
$ mplayer -nocache -afm ffmpeg http://wknc.sma.ncsu.edu:8000/wknchd1.mp3
3 command-line music players for Linux
cmus is a fast and flexible music player. It organizes your music by either musician or album title, and lets you load playlist files. You can also use cmus’ built-in search function to look for music in your /home directory and you can switch between several different layouts.
MOC (short for Music on Console) is a dual-pane music player that’s similar to cmus. The left pane lists the tracks in a folder, while the right pane contains the contents of a playlist file (if there is one in the folder).
If you want a quick and dirty way to play music files, then mpg123 and ogg123 are good options. I’ve lumped them together because both applications are very similar. In fact, the only difference between the two applications is the file formats they support.
ripit isn’t a music player, but it can help you feed your music player. It’s a script that works with a number of utilities and audio encoders—like cdparanoia, cdda2wav, Ogg Vobis, and LAME—to convert the tracks on a CD to files on your hard drive.
Graba tus sesiones de terminal en Linux con termtosvg
Termtosvg es una herramienta en linea de comandos que nos permite grabar nuestras sesiones de terminal en un entorno de animación SVG. Se trata de un formato soportado por los principales navegadores web (Chrome, Firefox que fue pionero a principios de esta década, Safari…), con la excepción del Edge de Microsoft, algo que a los linuxeros tampoco nos preocupa demasiado.
Creado en Python y con licencia libre BSD (3-Clause), termtosvg nos recuerda en diferentes aspectos a herramientas como script o el más reciente asciicinema, siempre una referencia a la hora de hacer demostraciones y tutoriales.
Shell parameter expansion: bash native regular expresssion for replacements
The ‘$’ character introduces parameter expansion, command substitution, or arithmetic expansion. The parameter name or symbol to be expanded may be enclosed in braces, which are optional but serve to protect the variable to be expanded from characters immediately following it which could be interpreted as part of the name.
When braces are used, the matching ending brace is the first ‘}’ not escaped by a backslash or within a quoted string, and not within an embedded arithmetic expansion, command substitution, or parameter expansion.
Setting WiFi up via the command line: Adding multiple wireless network configurations
network={
ssid=»HomeOneSSID»
psk=»passwordOne»
priority=1
id_str=»homeOne»
}
network={
ssid=»HomeTwoSSID»
psk=»passwordTwo»
priority=2
id_str=»homeTwo»
}
What is the difference between iface eth0 inet manual and iface eth0 inet static?
iface eth0 inet static
: Defines a static IP address for eth0
iface eth0 inet manual
: To create a network interface without an IP address at all. Usually used by interfaces that are bridge or aggregation members, or have a VLAN device configured on them