Presenting the Eshell

Demonstrated the iPython approach by entering the following into an ipython REPL:

Python 2.7.12 (default, Nov 19 2016, 06:48:10)
Type "copyright", "credits" or "license" for more information.

IPython 2.4.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: 2 ** 60
Out[1]: 1152921504606846976

In [2]: cd /tmp/testing
/tmp/testing

In [3]: ls
pi.py  pi.rb  README  src/  tests/

In [4]: cat pi.py
import math

print(math.pi)
print(math.cos(math.pi))

In [5]: chmod a+x pi.py
  File "<ipython-input-5-b9e30b9b8e31>", line 1
    chmod a+x pi.py
          ^
SyntaxError: invalid syntax


In [6]: run pi
3.14159265359
-1.0

In [7]: system ruby pi.rb
Out[7]: ['-0.9999999999964793', '-3.141592653589793']

In [8]: cat README
Ciao, all you cool cats.
Oh, and hey to all my dawgs.

In [9]: def cat(arg=None):
   ...:     return 'Meow!'
   ...:

In [10]: cat
Out[10]: <function __main__.cat>

In [11]: cat()
Out[11]: 'Meow!'

In [12]: cat README
  File "<ipython-input-12-01ccd53eccb4>", line 1
    cat README
             ^
SyntaxError: invalid syntax

In answer to your question, I haven’t looked to see why we have an array when calling the system function.

If you want to see what shell-like functions iPython has, type % and hit the Tab key, which shows something like:

In [13]: %
Display all 122 possibilities? (y or n)
%%!                      %doctest_mode            %pfile
%%HTML                   %ed                      %pinfo
%%SVG                    %edit                    %pinfo2
%%bash                   %env                     %popd
%%capture                %gui                     %pprint
%%debug                  %hist                    %precision
%%file                   %history                 %profile
%%html                   %install_default_config  %prun
%%javascript             %install_ext             %psearch
%%latex                  %install_profiles        %psource
%%perl                   %killbgscripts           %pushd
%%prun                   %ldir                    %pwd
%%pypy                   %less                    %pycat
%%python                 %lf                      %pylab
%%python2                %lk                      %quickref
%%python3                %ll                      %recall
%%ruby                   %load                    %rehashx
%%script                 %load_ext                %reload_ext
%%sh                     %loadpy                  %rep
%%svg                    %logoff                  %rerun
%%sx                     %logon                   %reset
%%system                 %logstart                %reset_selective
%%time                   %logstate                %rm
%%timeit                 %logstop                 %rmdir
%%writefile              %ls                      %run
%alias                   %lsmagic                 %save
%alias_magic             %lx                      %sc
%autocall                %macro                   %store
%autoindent              %magic                   %sx
%automagic               %man                     %system
%bookmark                %matplotlib              %tb
%cat                     %mkdir                   %time
%cd                      %more                    %timeit
%clear                   %mv                      %unalias
%colors                  %notebook                %unload_ext
%config                  %page                    %who
%cp                      %paste                   %who_ls
%cpaste                  %pastebin                %whos
%debug                   %pdb                     %xdel
%dhist                   %pdef                    %xmode
%dirs                    %pdoc

At this point, we start an eshell process, and demonstrate some of the shell-like features that we’d expect for something that ends with -shell:

$ cd /tmp/testing

$ pwd
/tmp/testing

$ ls
README  pi.py  pi.rb  src  tests

$ cat README
Ciao, all you cool cats.
Oh, and hey to all my dawgs.

$ ruby pi.rb
-0.9999999999964793
-3.141592653589793

$ python *.py
3.141592653589793
-1.0

$ echo "Hello"
Hello

$ echo 'Hello'
Hello

$ echo Hello
Hello

$ alias ee 'find-file-other-window $1'

$ ee pi.rb
#<buffer pi.rb>

This last example shows that ee opens a window in another window. Note, however, that the alias is actually calling an Emacs function, not another executable (although it could).

To demonstrate how eshell is a REPL, let’s type a simple Lisp expression:

$ (length "hi")
2

$ length "hi"      # ← Works since shell parser calls Lisp
2

$ length hi        # ← Works since shell reads as string
2

$ (+ 1 3)
4

$ + 1 3            # ← Works since shell reads as number
4

$ * 3 (+ 1 2)      # ← Shows both shell and lisp parsers
9

$ ls
README  pi.py  pi.rb  src  tests

$ length *        # Globs return a list
5

$ length *.py
1

$ touch 'and go.py'

$ ls *.py
and go.py  pi.py

$ echo *.py        # ← More clear that globs are lists
("and go.py" "pi.py")

To drive home the differences between shell and lisp parsers, let’s enter the following in eshell buffer:

$ setq ANSWER 42   # ← Normal Emacs variable
42

$ numberp $ANSWER  # ← Use $ to get value.
t

$ setq UNANSWER "41"
41

$ stringp $UNANSWER
t

$ mod ANSWER 5    # ← Forgot the $ with shell parser
Wrong type argument: number-or-marker-p, "ANSWER"

$ mod $ANSWER 5   # ← Math without expr
2

$ (mod ANSWER 5)  # ← Lisp doesn't need $ for vars
2

$ (mod $ANSWER 5)
Symbol's value as variable is void: $ANSWER

$ echo $UNANSWER:$ANSWER
41:42

$ echo $UNANSWER:(mod ANSWER 5)   # ← Let's talk about predicate filters later in the show
Malformed modification time modifier `m'

$ echo $UNANSWER:$(mod ANSWER 5)
41:2

$ echo $ANSWER:${mod $ANSWER 5}
42:2

See this page for more details.

Show off the for concept, by entering the following in an eshell buffer. Note that the do syntax for some shells doesn’t work. Loops look more like csh’s:

$ for F in *; do echo "I like $F"; done
Symbol's value as variable is void: do
done: command not found

$ for F in * { echo "I like $F" }
I like README
I like and go.py
I like pi.py
I like pi.rb
I like src/
I like tests/

# A list can be generated in any way, like with Lisp:
$ for N in (number-sequence 1 5) { % $ANSWER $N }
0
0
0
2
2

# Generate the list with eshell parser syntax:
$ for N in {number-sequence 1 5} { % $ANSWER $N }
0
0
0
2
2

# Note: Can't use Lisp as the loop's action:
$ for N in {number-sequence 1 5} (% ANSWER N)

# Unless you embed the Lisp in shell parser:
$ for N in {number-sequence 1 5} {(% ANSWER N)}
0
0
0
2
2

To demonstrate the precedence order for eshell commands, I created a script called foobar that simply contains:

#!/bin/sh

echo "Called: executable"

Without anything else, this will be called:

$ which foobar
/home/howard/bin/foobar

$ foobar
Called: executable

We now create a regular Emacs function in Lisp (notice that it isn’t interactive):

(defun foobar ()
  "Called: function")

It now takes precedence over the executable:

$ which foobar
foobar is a Lisp function

$ foobar
Called: function

Create another Lisp function, this has the eshell/ prefix. Again, no need to make interactive:

(defun eshell/foobar ()
  "Called: eshell function")

And this new function over-shadows the others:

$ which foobar
eshell/foobar is a Lisp function

$ foobar
Called: eshell function

Finally, we define an alias, and demonstrate that it over-shadows all the others:

$ alias foobar 'echo "Called: alias"'

$ which foobar
foobar is an alias, defined as "echo "Called: alias""

$ foobar
Called: alias

Using a directory for this purpose, we can demonstrate EShell’s predicate filter feature. First, list all files:

$ ls *(.)
README  and go.py  pi.py  pi.rb

$ ls *(^/)   # ← Inverse of directories are often files
README  and go.py  pi.py  pi.rb

Demonstrate combining modifiers by listing all files with more than 50 bytes to them:

$ ls *(.L+50)
README  pi.py

After creating three files (using the touch executable), we can list all empty files (that is, those that have less than 1 byte):

$ ls *(L-1)
and go.py  goo.py  grip.py  swam.py

Or those modified less than 40 seconds ago:

$ ls *(.ms-40)
README  and go.py  goo.py  grip.py  pi.py  pi.rb  src  swam.py  tests

Modified after we modified goo.py:

$ ls *(.m-'goo.py')
grip.py  swam.py

And before we modified goo.py:

$ ls *(.m+'goo.py')
README  and go.py  pi.py  pi.rb

I love this. I can get a list of my journal entries larger than 5000 bytes, and open dired showing only those files:

$ dired ~/journal/2017*(L+5000)

Convert a string with a modifier:

$ echo "hello"(:U)
HELLO

Convert all files, as strings, in a list:

$ echo *(:U)
("README" "AND GO.PY" "GOO.PY" "GRIP.PY" "PI.PY" "PI.RB" "SRC/" "SWAM.PY" "TESTS/")

Modifiers can be combined with filters:

$ echo *(.L-1:U)
("AND GO.PY" "GOO.PY" "GRIP.PY" "SWAM.PY")

However, we often split these and use the for loop:

$ for F in *(.L-1) { mv $F $F(:U) }

$ ls
AND GO.PY  GOO.PY  GRIP.PY  README  SWAM.PY  pi.py  pi.rb  src  tests

Now, all empty files are in upper case.

You’d think you could reverse a list with:

$ echo ("hello" "cruel" "world")(:R)
No matches found: ("hello" "cruel" "world")

Since the shell parser doesn’t like that syntax, perhaps you could set the list to a variable and work with that?

$ setq BADDABING (list "hello" "cruel" "world")
("hello" "cruel" "world")

$ echo $BADDABING(:R)
("world" "cruel" "hello")

If you find this stuff too odd and confusing, you can always fall back to Lisp:

$ reverse (list "hello" "cruel" "world")
("world" "cruel" "hello")

• Functions for Eshell: eshell/
• They do not need to be interactive

• Functions should assume &rest for arguments:

  (defun eshell/do-work (&rest args)
    "Do some work in an optional directory."
    (let ((some-dir (if args
                         (pop args)
                      default-directory)))
      (message "Work in %s" some-dir)))
  

To have eshell work on a remote server:

(let ((default-directory "/ssh:your-host.com:public/"))
  (eshell))

My personal project:

• Connect to my hypervisor controller
• Download and store a list of virtual machines
• Use ido-completing-read to select a host / ip
• Generate a Tramp URL for default-directory

The User predicate (U) could have been written:

(defun file-owned-current-uid-p (file)
  (when (file-exists-p file)
      (= (nth 2 (file-attributes file))
         (user-uid))))

Then add it:

(add-hook 'eshell-pred-load-hook (lambda ()
  (add-to-list 'eshell-predicate-alist
           '(?U . 'file-owned-current-uid-p))))

My engineering notebook is a directory of files.

Most of my files have #+tags entries.

I can filter based on these tag entries.

I have to parse text following predicate key.

Pipes for shell are flexible, but…

• Shell’s text processing is limited
• Need arsenal of tiny, cryptic programs
• Re-run many times since debug pipe steps

Emacs is pretty good at text processing

• keep-lines / flush-lines instead of grep
• replace-string, et. al instead of sed

In EShell, redirect output to Emacs buffer:

$ some-command > #<buffer buf-name>

After editing the buffer, use it:

$ bargs #<buf-name> mv % /tmp/testing

Reference buffers as #<buf-name> with:

(setq eshell-buffer-shorthand t)

Or use keybinding, C-c M-b

Initial implementation of bargs:

(defun eshell/-buffer-as-args (buffer separator command)
  "Takes the contents of BUFFER, and splits it on SEPARATOR, and
runs the COMMAND with the contents as arguments. Use an argument
`%' to substitute the contents at a particular point, otherwise,
they are appended."
  (let* ((lines (with-current-buffer buffer
                  (split-string
                   (buffer-substring-no-properties (point-min) (point-max))
                   separator)))
         (subcmd (if (-contains? command "%")
                     (-flatten (-replace "%" lines command))
                   (-concat command lines)))
         (cmd-str  (string-join subcmd " ")))
    (message cmd-str)
    (eshell-command-result cmd-str)))

(defun eshell/bargs (buffer &rest command)
  "Passes the lines from BUFFER as arguments to COMMAND."
  (eshell/-buffer-as-args buffer "\n" command))

(defun eshell/sargs (buffer &rest command)
  "Passes the words from BUFFER as arguments to COMMAND."
  (eshell/-buffer-as-args buffer nil command))