VisiData

A very useful tool for data wrangling.

check this video: VisiData Lightning Demo at PyCascades 2018 - YouTube

---

$ pip install visidata

some quick test:

$ ps aux | vd

$ ls -l | vd -f fixed

To load another file, press o and enter a filepath.

Press S to access other loaded or derived sheets.

use < and > to move through a categorical column.

basic movement: h, j, k and l, add g before to move all the way, for example gj move to bottom

/ or ? to search (slow?)

select rows

s to select, u to unselect, t for toggling

add g in front for all, for example gs to select all

| to select by regex, `‘ to unselect by regex rows matching in current column

add g to select/unselect rows match any visible columns

, select rows matching current cell in current column

g, select rows matching currnet row in all visible columns

after selected rows

gz to open a duplicate sheet with copies of the selected rows.

• d to delete and y to yank the row
• p/P to paste the row after/before current row

add g to apply all, for example gd to delete selected rows

sort rows

type the column:

• #: int
• %: float
• @: date
• ~: string
• $: currency

[ or ] sorts ascending/descending by current column

sort by multiple rows

1. Press ! on those columns to designate them as key columns.
2. Press g[ or g] to sort.

columns

! pins the current column on the left as a key column

H or L slides the current column one position to the left/right (add g to move to all the way)

- (hyphen) to hide the current column.

unhide:

• C to open columns sheet
• move to the 0 cell, e to change to a positive value
• q to quit current sheet and return to original sheet

type a column:

• #: int
• %: float
• @: date
• ~: string
• $: currency

type multiple columns:

• C to open columns sheet
• s to select
• g~ to type selected columns to string type
• q to quit

split a column:

• :: adds new columns derived from splitting the current column at positions defined by a regex pattern.
• ;: adds new columns derived from pulling the contents of the current column which match the regex within capture groups.
• *: followed by regex/substring replaces the text which matches the capture groups in regex with the contents of substring. substring may include backreferences (\1 etc).

concatenate columns:

1. Type = followed by Year + '-' + Month + '-' + Day
2. Set the type of the new derived column by pressing @ (date).
3. Type ^ followed by Date to rename the column to Date.

aggregation:

• type the column
• select rows
• Type g+ followed by avg

edit content

• e: edits contents of current cell
• ge text: sets contents of current column for selected rows to text

in edit mode:

• Enter: accepts input
• Ctrl-C or Esc: aborts input
• Ctrl-O: opens external $EDITOR to edit contents
• Ctrl-R: reloads initial value
• Up or Down: sets contents to previous/next in history
• Tab or Shift-Tab: autocompletes input (when available)

rename columns:

• ^: edits name of current column
• g^: sets names of all unnamed visible columns to contents of selected rows (or current row)
• z^: sets name of current column to contents of current cell
• gz^: sets name of current column to combined contents of current column for selected rows

aggregation

+ aggregator: adds aggregator to current column z+ aggregator: displays result of aggregator over values in selected rows for current column

aggregators:

• min: smallest value in the group
• max: largest value in the group
• avg/mean: average value of the group
• median: median value in the group
• sum: total summation of all numbers in the group
• distinct: number of distinct values in the group
• count: number of values in the group
• keymax: key of the row with the largest value in the group

pivot table

1. Press ! on the independent variable to set it as a key column.
2. Press + on a second column to add an aggregrate. This configures which column is to be composed in relation to others.
3. Optional: Set the type for the column
4. Press W on the dependent categorical variable to pivot on it.

F to open the Frequency table.

graphs

. (dot=plot) on that column

interact with graphs: http://visidata.org/docs/graph/