338 lines
No EOL
10 KiB
Markdown
338 lines
No EOL
10 KiB
Markdown
+++
|
||
title = "Doce herramientas esenciales en línea de comandos para cualquier científicx de datos"
|
||
date = 2020-01-01T00:00:00+01:00
|
||
lastmod = 2022-10-09T19:09:37+02:00
|
||
draft = false
|
||
subtitle = "12 Essential Command Line Tools for Data Scientist"
|
||
math = true
|
||
diagram = true
|
||
authors = ["adolfo_anton"]
|
||
[image]
|
||
caption = "Captura de algo"
|
||
focal_point = "BottomLeft"
|
||
placement = 3
|
||
preview_only = false
|
||
+++
|
||
|
||
Este artículo, publicado originalmente en KDNuggets por [Matthew Mayo](https://www.dataquest.io/blog/author/matthew-mayo/) y republicado [por Dataquest](https://www.dataquest.io/blog/top-12-essential-command-line-tools-for-data-scientists/), lo traduzco sin literalidad en aras de una mayor comprensión.
|
||
|
||
Se considera un rápido vistazo a una docena de herramientas de línea de comandos para sistemas compatibles con el estándar POSIX que pueden ser útiles para tareas realizadas en ciencia de datos.
|
||
|
||
La lista no incluye comandos de gestión de archivos como `pwd`, `ls`, `mkdir`, `rm` o herramientas de gestión de sesiones remotas como `rsh` o `ssh`. Pero sí que son utilidades desde la perspectiva de la ciencia de datos, normalmente en inspección y procesamiento de datos. Suelen estar incluidas en los sistemas operativos compatibles con POSIX. Son ejemplos muy elementales que os anima a complementar por vuestra cuenta cuando sea necesario. En este caso no menciona los manuales tradicionales de estas herramientas sino las entradas de la Wikipedia que considera más amigables para recién llegadxs.
|
||
|
||
|
||
## 1. wget {#1-dot-wget}
|
||
|
||
`wget` es una herramienta para la recuperación (obtención) de archivos, normalmente de recursos remotos. Su uso básico sirve para descargar un archivo, `iris.csv`, un archivo de datos con el que se va a jugar en el resto de los comandos:
|
||
|
||
```bash
|
||
wget https://raw.githubusercontent.com/uiuc-cse/data-fa14/gh-pages/data/iris.csv
|
||
```
|
||
|
||
```bash { linenos=true, linenostart=1 }
|
||
ls iris.csv
|
||
```
|
||
|
||
|
||
## 2. cat {#2-dot-cat}
|
||
|
||
`cat` es una herramienta para mostrar contenidos de archivos en la salida estándar o `STDOUT` y concatenar varios archivos, pudiendo combinarlos, añadirlos al final, numerar las líneas…
|
||
|
||
Una opción es ver el archivo anterior:
|
||
|
||
```bash { linenos=true, linenostart=1 }
|
||
cat iris.csv
|
||
```
|
||
|
||
```bash
|
||
sepal_length,sepal_width,petal_length,petal_width,species
|
||
5.1,3.5,1.4,0.2,setosa
|
||
4.9,3,1.4,0.2,setosa
|
||
4.7,3.2,1.3,0.2,setosa
|
||
4.6,3.1,1.5,0.2,setosa
|
||
5,3.6,1.4,0.2,setosa
|
||
5.4,3.9,1.7,0.4,setosa
|
||
4.6,3.4,1.4,0.3,setosa
|
||
5,3.4,1.5,0.2,setosa
|
||
4.4,2.9,1.4,0.2,setosa
|
||
4.9,3.1,1.5,0.1,setosa
|
||
5.4,3.7,1.5,0.2,setosa
|
||
4.8,3.4,1.6,0.2,setosa
|
||
4.8,3,1.4,0.1,setosa
|
||
4.3,3,1.1,0.1,setosa
|
||
5.8,4,1.2,0.2,setosa
|
||
5.7,4.4,1.5,0.4,setosa
|
||
5.4,3.9,1.3,0.4,setosa
|
||
5.1,3.5,1.4,0.3,setosa
|
||
5.7,3.8,1.7,0.3,setosa
|
||
5.1,3.8,1.5,0.3,setosa
|
||
5.4,3.4,1.7,0.2,setosa
|
||
5.1,3.7,1.5,0.4,setosa
|
||
4.6,3.6,1,0.2,setosa
|
||
5.1,3.3,1.7,0.5,setosa
|
||
4.8,3.4,1.9,0.2,setosa
|
||
5,3,1.6,0.2,setosa
|
||
5,3.4,1.6,0.4,setosa
|
||
5.2,3.5,1.5,0.2,setosa
|
||
5.2,3.4,1.4,0.2,setosa
|
||
4.7,3.2,1.6,0.2,setosa
|
||
4.8,3.1,1.6,0.2,setosa
|
||
5.4,3.4,1.5,0.4,setosa
|
||
5.2,4.1,1.5,0.1,setosa
|
||
5.5,4.2,1.4,0.2,setosa
|
||
4.9,3.1,1.5,0.1,setosa
|
||
5,3.2,1.2,0.2,setosa
|
||
5.5,3.5,1.3,0.2,setosa
|
||
4.9,3.1,1.5,0.1,setosa
|
||
4.4,3,1.3,0.2,setosa
|
||
5.1,3.4,1.5,0.2,setosa
|
||
5,3.5,1.3,0.3,setosa
|
||
4.5,2.3,1.3,0.3,setosa
|
||
4.4,3.2,1.3,0.2,setosa
|
||
5,3.5,1.6,0.6,setosa
|
||
5.1,3.8,1.9,0.4,setosa
|
||
4.8,3,1.4,0.3,setosa
|
||
5.1,3.8,1.6,0.2,setosa
|
||
4.6,3.2,1.4,0.2,setosa
|
||
5.3,3.7,1.5,0.2,setosa
|
||
5,3.3,1.4,0.2,setosa
|
||
7,3.2,4.7,1.4,versicolor
|
||
6.4,3.2,4.5,1.5,versicolor
|
||
6.9,3.1,4.9,1.5,versicolor
|
||
5.5,2.3,4,1.3,versicolor
|
||
6.5,2.8,4.6,1.5,versicolor
|
||
5.7,2.8,4.5,1.3,versicolor
|
||
6.3,3.3,4.7,1.6,versicolor
|
||
4.9,2.4,3.3,1,versicolor
|
||
6.6,2.9,4.6,1.3,versicolor
|
||
5.2,2.7,3.9,1.4,versicolor
|
||
5,2,3.5,1,versicolor
|
||
5.9,3,4.2,1.5,versicolor
|
||
6,2.2,4,1,versicolor
|
||
6.1,2.9,4.7,1.4,versicolor
|
||
5.6,2.9,3.6,1.3,versicolor
|
||
6.7,3.1,4.4,1.4,versicolor
|
||
5.6,3,4.5,1.5,versicolor
|
||
5.8,2.7,4.1,1,versicolor
|
||
6.2,2.2,4.5,1.5,versicolor
|
||
5.6,2.5,3.9,1.1,versicolor
|
||
5.9,3.2,4.8,1.8,versicolor
|
||
6.1,2.8,4,1.3,versicolor
|
||
6.3,2.5,4.9,1.5,versicolor
|
||
6.1,2.8,4.7,1.2,versicolor
|
||
6.4,2.9,4.3,1.3,versicolor
|
||
6.6,3,4.4,1.4,versicolor
|
||
6.8,2.8,4.8,1.4,versicolor
|
||
6.7,3,5,1.7,versicolor
|
||
6,2.9,4.5,1.5,versicolor
|
||
5.7,2.6,3.5,1,versicolor
|
||
5.5,2.4,3.8,1.1,versicolor
|
||
5.5,2.4,3.7,1,versicolor
|
||
5.8,2.7,3.9,1.2,versicolor
|
||
6,2.7,5.1,1.6,versicolor
|
||
5.4,3,4.5,1.5,versicolor
|
||
6,3.4,4.5,1.6,versicolor
|
||
6.7,3.1,4.7,1.5,versicolor
|
||
6.3,2.3,4.4,1.3,versicolor
|
||
5.6,3,4.1,1.3,versicolor
|
||
5.5,2.5,4,1.3,versicolor
|
||
5.5,2.6,4.4,1.2,versicolor
|
||
6.1,3,4.6,1.4,versicolor
|
||
5.8,2.6,4,1.2,versicolor
|
||
5,2.3,3.3,1,versicolor
|
||
5.6,2.7,4.2,1.3,versicolor
|
||
5.7,3,4.2,1.2,versicolor
|
||
5.7,2.9,4.2,1.3,versicolor
|
||
6.2,2.9,4.3,1.3,versicolor
|
||
5.1,2.5,3,1.1,versicolor
|
||
5.7,2.8,4.1,1.3,versicolor
|
||
6.3,3.3,6,2.5,virginica
|
||
5.8,2.7,5.1,1.9,virginica
|
||
7.1,3,5.9,2.1,virginica
|
||
6.3,2.9,5.6,1.8,virginica
|
||
6.5,3,5.8,2.2,virginica
|
||
7.6,3,6.6,2.1,virginica
|
||
4.9,2.5,4.5,1.7,virginica
|
||
7.3,2.9,6.3,1.8,virginica
|
||
6.7,2.5,5.8,1.8,virginica
|
||
7.2,3.6,6.1,2.5,virginica
|
||
6.5,3.2,5.1,2,virginica
|
||
6.4,2.7,5.3,1.9,virginica
|
||
6.8,3,5.5,2.1,virginica
|
||
5.7,2.5,5,2,virginica
|
||
5.8,2.8,5.1,2.4,virginica
|
||
6.4,3.2,5.3,2.3,virginica
|
||
6.5,3,5.5,1.8,virginica
|
||
7.7,3.8,6.7,2.2,virginica
|
||
7.7,2.6,6.9,2.3,virginica
|
||
6,2.2,5,1.5,virginica
|
||
6.9,3.2,5.7,2.3,virginica
|
||
5.6,2.8,4.9,2,virginica
|
||
7.7,2.8,6.7,2,virginica
|
||
6.3,2.7,4.9,1.8,virginica
|
||
6.7,3.3,5.7,2.1,virginica
|
||
7.2,3.2,6,1.8,virginica
|
||
6.2,2.8,4.8,1.8,virginica
|
||
6.1,3,4.9,1.8,virginica
|
||
6.4,2.8,5.6,2.1,virginica
|
||
7.2,3,5.8,1.6,virginica
|
||
7.4,2.8,6.1,1.9,virginica
|
||
7.9,3.8,6.4,2,virginica
|
||
6.4,2.8,5.6,2.2,virginica
|
||
6.3,2.8,5.1,1.5,virginica
|
||
6.1,2.6,5.6,1.4,virginica
|
||
7.7,3,6.1,2.3,virginica
|
||
6.3,3.4,5.6,2.4,virginica
|
||
6.4,3.1,5.5,1.8,virginica
|
||
6,3,4.8,1.8,virginica
|
||
6.9,3.1,5.4,2.1,virginica
|
||
6.7,3.1,5.6,2.4,virginica
|
||
6.9,3.1,5.1,2.3,virginica
|
||
5.8,2.7,5.1,1.9,virginica
|
||
6.8,3.2,5.9,2.3,virginica
|
||
6.7,3.3,5.7,2.5,virginica
|
||
6.7,3,5.2,2.3,virginica
|
||
6.3,2.5,5,1.9,virginica
|
||
6.5,3,5.2,2,virginica
|
||
6.2,3.4,5.4,2.3,virginica
|
||
5.9,3,5.1,1.8,virginica
|
||
```
|
||
|
||
|
||
## 3. wc {#3-dot-wc}
|
||
|
||
`wc` viene de _word counts_ y sirve para contar palabras, líneas o _bytes_ de archivos de texto. Sin argumentos muestra el número de líneas, palabras, caracteres y el nombre del archivo.
|
||
|
||
```bash
|
||
wc iris.csv
|
||
```
|
||
|
||
|
||
## 4. head {#4-dot-head}
|
||
|
||
`head` muestra las primeras diez líneas de un archivo si no se especifican otras opciones. Se puede cambiar el número con la opción `-n`.
|
||
|
||
```bash
|
||
head -n 5 iris.csv
|
||
```
|
||
|
||
|
||
## 5. tail {#5-dot-tail}
|
||
|
||
Any guesses as to what tail does?
|
||
|
||
~$ tail -n 5 iris.csv
|
||
|
||
6.7,3,5.2,2.3,virginica
|
||
6.3,2.5,5,1.9,virginica
|
||
6.5,3,5.2,2,virginica
|
||
6.2,3.4,5.4,2.3,virginica
|
||
5.9,3,5.1,1.8,virginica
|
||
1K407sZ
|
||
|
||
Working that command line sorcery.
|
||
|
||
|
||
## 6. find {#6-dot-find}
|
||
|
||
find is a utility for searching the file system for particular files. The following searches the tree structure starting in the current directory (“.”) for any file starting with “iris” and ending in any dumber of characters (“-name ‘iris\*'”) of regular file type (“-type f”):
|
||
|
||
~$ find . -name 'iris\*' -type f
|
||
|
||
./iris.csv
|
||
./notebooks/kmeans-sharding-init/sharding/tests/results/iris_time_results.csv
|
||
./notebooks/ml-workflows-python-scratch/iris_raw.csv
|
||
./notebooks/ml-workflows-python-scratch/iris_clean.csv
|
||
…
|
||
|
||
|
||
## 7. cut {#7-dot-cut}
|
||
|
||
cut is used for slicing out sections of a line of text from a file. While these slices can be made using a variety of criteria, cut can be useful for extracting columnar data from CSV files. This outputs the fifth column (“-f 5”) of the iris.csv file using the comma as field delimiter (“-d ‘,'”):
|
||
|
||
~$ cut -d ',' -f 5 iris.csv
|
||
species
|
||
setosa
|
||
setosa
|
||
setosa
|
||
…
|
||
|
||
|
||
## 8. uniq {#8-dot-uniq}
|
||
|
||
uniq modifies the output of text files to standard output by collapsing identical consecutive lines into a single copy. On its own, this may not seem too terribly interesting, but when used to build pipelines at the command line (piping the output of one command into the input of another, and so on), this can become useful.
|
||
|
||
The following gives us a unique count of the iris dataset class names held in the fifth column, along with their counts:
|
||
|
||
~$ tail -n 150 iris.csv | cut -d "," -f 5 | uniq -c
|
||
|
||
50 setosa
|
||
50 versicolor
|
||
50 virginica
|
||
cowsay-data-science
|
||
|
||
What the cow say.
|
||
|
||
|
||
## 9. awk {#9-dot-awk}
|
||
|
||
awk isn’t actually a “command,” but is instead a full programming language. It is meant for processing and extracting text, and can be invoked from the command line in single line command form.
|
||
|
||
Mastery of awk would take some time, but until then here is a sample of what it can accomplish. Considering that our sample file — iris.csv — is rather limited (especially when it comes to diversity of text), this line will invoke awk, search for the string “setosa” within a given file (“iris.csv”), and print to standard output, one by one, the items which it has encountered (held in the $0 variable):
|
||
|
||
~$ awk '_setosa_ { print $0 }' iris.csv
|
||
|
||
5.1,3.5,1.4,0.2,setosa
|
||
4.9,3,1.4,0.2,setosa
|
||
4.7,3.2,1.3,0.2,setosa
|
||
4.6,3.1,1.5,0.2,setosa
|
||
5,3.6,1.4,0.2,setosa
|
||
|
||
|
||
## 10. grep {#10-dot-grep}
|
||
|
||
grep is another text processing tool, this one for string and regular expression matching.
|
||
|
||
~$ grep -i "vir" iris.csv
|
||
|
||
6.3,3.3,6,2.5,virginica
|
||
5.8,2.7,5.1,1.9,virginica
|
||
7.1,3,5.9,2.1,virginica
|
||
…
|
||
If you spend much time doing text processing at the command line, grep is definitely a tool you will get to know well. See here for some more useful details.
|
||
|
||
|
||
## 11. sed {#11-dot-sed}
|
||
|
||
sed is a stream editor, yet another text processing and transformation tool, similar to awk. Let’s use it below to change the occurrences of “setosa” in our iris.csv file to “iris-setosa,” using this line:
|
||
|
||
~$
|
||
|
||
sed 's/setosa/iris-setosa/g' iris.csv > output.csv
|
||
~$ head output.csv
|
||
|
||
sepal_length,sepal_width,petal_length,petal_width,species
|
||
5.1,3.5,1.4,0.2,iris-setosa
|
||
4.9,3,1.4,0.2,iris-setosa
|
||
4.7,3.2,1.3,0.2,iris-setosa
|
||
…
|
||
|
||
|
||
## 12. history {#12-dot-history}
|
||
|
||
history is pretty straightforward, but also pretty useful, especially if you’re depending on replicating some data preparation you accomplished at the command line.
|
||
|
||
~$ history
|
||
|
||
547 tail iris.csv
|
||
548 tail -n 150 iris.csv
|
||
549 tail -n 150 iris.csv | cut -d "," -f 5 | uniq -c
|
||
550 clear
|
||
551 history
|
||
|
||
|
||
## Conclusión {#conclusión}
|
||
|
||
Estos doce comandos se dan en el módulo de [Fundamentos tecnológicos](https://mpvd.es/modulo/01-fundamentos-tecnologicos/) pero, como decía el artículo original, son solo una muestra de lo que es posible hacer en ciencia de datos desde la línea de comandos.
|
||
|
||
Finaliza el artículo con un lema que podría titular el artículo: libérate del ratón y mira cómo crece tu productividad. |