------------------------------------------------------------------------- PHYS210 UNIX 2, 3 and 4 LABS SUMMARY homework directories, startup Files, aliases ssh, shell variables, bash pattern matching, history standard input and standard output, input/output redirection pipes, grep and regular expressions all those quotes ------------------------------------------------------------------------- 1) THE /phys210 DIRECTORY AND YOUR SUB-DIRECTORY WITHIN IT Login and locate your personal directory within the /phys210 directory % cd /phys210/your-login-name or % cd /phys210/$LOGNAME The latter command will work for everyone. NOTE! In the following, whenever you see something that you are to type that includes $LOGNAME, you can replace $LOGNAME with your login (account) name The instructor will be doing this demo as user 'phys210d' % pwd /phys210/phys210d Get a long listing of the directory % ls -lt total 20 drwx------ 2 phys210d ugrad 4096 Aug 29 16:15 hw1/ drwx------ 2 phys210d ugrad 4096 Aug 29 16:15 hw2/ drwx------ 2 phys210d ugrad 4096 Aug 29 16:15 hw3/ drwx------ 2 phys210d ugrad 4096 Aug 29 16:15 hw4/ drwxr-xr-x 2 phys210d ugrad 4096 Aug 29 16:15 public_html/ Note the permissions on hw[1-4], NO access except by user (i.e. you). Please keep them that way. Change to the public_html directory, When *you* type pwd following execution of the cd command, you should see /phys210/$LOGNAME/public_html % cd public_html % pwd /phys210/phys210d/public_html Get a directory listing, should be a single file 'index.html' % ls index.html View the contents of the file: in your case you should see the text $LOGNAME % cat index.html phys210d You will be replacing/modifying index.html in Problem 2 of Homework 1. Access /phys210/$LOGNAME/public_html/index.html using a browser as follows. Point the browser to http://laplace.physics.ubc.ca/Students/$LOGNAME e.g., in my case http://laplace.physics.ubc.ca/Students/phys210d Again, you should see the text $LOGNAME in the browser window. ------------------------------------------------------------------------- 2) INSTALLING THE DEFAULT 210 BASH STARTUP FILES Change to your home directory % cd Look for the following files .bash_profile .bash_login .profile .bashrc .aliases using % ls .bash_profile % ls .bash_login % ls .profile % ls .bashrc % ls .aliases For any that exist, and unless you're a Linux expert and you know what these files do, and you're happy with their contents, then move them as follows, and as necessary, using one or more of the following mv commands *** % cp -i .bash_profile .bash_profile.O % cp -i .bash_login .bash_login.O % cp -i .profile .profile.O % cp -i .bashrc .bashrc.O % cp -i .aliases .aliases.O In the highly unlikely case that any of the .O files alrneady exist, then back *them* up using, for example % mv -i .bashrc.O .bashrc.OO, and re-execute % mv -i .bashrc .bashrc.O Now copy the PHYS 210 default startup files from the phys210 account to your home directory. Don't confuse phys210's home, referred to as ~phys210 or /home/phys210 with the /phys210 directory discussed in 1. and in which you will do your homework! Again, ensure that you are in your home directory. When *you* type pwd following execution of cd, you should of course see /home/$LOGNAME % cd % pwd /home/phys210d Do the copying, AND BE CAUTIOUS; i.e. use the -i option to cp. If cp prompts you to confirm overwrite then you have NOT mv'ed the original file(s) per the above instructions, so go back to *** above and try mv'ing the file(s) again. % cp -i ~phys210/.bashrc . % cp -i ~phys210/.profile . % cp -i ~phys210/.aliases . Now start up a NEW TERMINAL WINDOW and display the aliases that are defined % alias # You should see output like the following alias CP='/bin/cp' alias LN='/bin/ln' alias MV='/bin/mv' alias RM='/bin/rm' alias a='alias' alias cp='cp -i' alias hi='history' alias ln='ln -i' alias ls='ls --color=auto -CF' alias mv='mv -i' alias rm='rm -i' alias xmaple='xmaple -cw' Display your path: /home/$LOGNAME/path should appear in it, as well as '.' (the current directory) which means the shell will always look in the current directory for commands to execute % echo $PATH .:/home/phys210d/bin:/home/phys210/bin:/etc:/usr/etc:/usr/ucb:/bin:/usr/bin:/usr/local/bin:/opt/maple16/bin As alternative way to display the value of the PATH environment variable % printenv PATH .:/home/phys210d/bin:/home/phys210/bin:/etc:/usr/etc:/usr/ucb:/bin:/usr/bin:/usr/local/bin ------------------------------------------------------------------------- 3) DEFINING ALIASES Add a new alias to your ~/.aliases file It's a bit inconvenient to have to type % cd /phys210/$LOGNAME/hw1 to get to your Homework 1 directory, so let's define an alias that will take you there with less typing First, cd to your home directory, and make a back up copy of your ~/.aliases file (making back ups before you start modifying important files is an extremely good habit to develop!) % cd % cp .aliases .aliases.O And verify that you *do* have a back ups, as well as the original % ls .aliases .aliases.O .aliases .aliases.O Now, open ~/.aliases (/home/$LOGNAME/.aliases) with your text editor of choice, and add the following four lines at the end of the file (if you're having trouble opening /home/$LOGNAME/.aliases, ask for help!) #----------------------------------------------------------------------- # Define an alias to cd to Homework 1 directory #----------------------------------------------------------------------- alias cdhw1="cd /phys210/$LOGNAME/hw1" Once you have (carefully!) entered these 4 lines, save the file---your editor may prompt you to confirm overwrite of the existing file /home/$LOGNAME/.aliases, but since you have a back ups, it's safe to do the overwriting. Note that due to the use of double quotes in the alias definition (we'll discuss this in a future lecture), you can use either $LOGNAME literally, or use your actual login (account) name. Now source ~/.aliases to "activate" the new alias ... % source ~/.aliases ... verify that 'cdhw1' *has* been defined as an alias ... % alias cdhw1 alias cdhw1='cd /phys210/phys210d/hw1' ... and try it out % cdhw1 % pwd /phys210/phys210d/hw1 ------------------------------------------------------------------------- 4) EXERCISE! Add additional alias definitions to ~/.aliases to change to your 2nd through 4th homework directories, as well as to cd to /phys210/$LOGNAME/public_html. When you're done, be sure to source ~/.aliases, or start a new terminal window, and check that your new definitions have been made and work. IMPORTANT!! DON'T FORGET TO MAKE A BACK UP OF ~/.aliases BEFORE YOU START MODIFYING IT. This may mean overwriting the old back up, but if everything so far has worked as advertised, that should be OK. ------------------------------------------------------------------------- 5) HOMEWORK 1 In principle, at this point you should be able to complete Problems 1 and 2, although you may have to do a little supplementary study via web resources, using the help facility or other documentation for your text editor of choice etc. ------------------------------------------------------------------------- 6) SSH Open a terminal window and type the following % ssh $LOGNAME@hyper.physics.ubc.ca where you can either use $LOGNAME literally, or replace it with your account name. You should see something like the following: The authenticity of host 'hyper.physics.ubc.ca (142.103.236.100)' can't be established. RSA key fingerprint is c2:eb:96:0f:17:b1:73:b5:86:3c:61:f8:b9:92:f0:85. Are you sure you want to continue connecting (yes/no)? This is a standard error message that ssh will emit whenever you try connecting to a new host--it's a safeguard whose details we will ignore for the time being, so answer yes to the question. You will then be asked for your password---enter the same one that you use to access the lab machines. phys210d@hyper.physics.ubc.ca's password: Provided you input the correct password, you should then see something like Last login: Thu Aug 30 16:57:57 2012 from bh0.physics.ubc.ca Execute the following command to verify that you are indeed logged into hyper: % hostname hyper Once you have done this, type % exit This will close the connection to hyper, and control will be returned to your shell on your lab machine On hyper, you have the same home directory as on the lab machines, and have access to all of the files therein, as well as /phys210/$LOGNAME and all of the files there as well. As discussed in the class notes, ssh-ing to hyper will be most useful should you want to connect from, e.g., a shell running on your home machine. Also note that if you don't specify the login name in a ssh command, it is assumed that you want to login using $LOGNAME. The first usage is *necessary* should you wish to access an account on a remote machine that does not have the same name as the account from which you are issuing the ssh command. Finally, as again discussed in the notes, ssh can be used to execute one or more commands on the remote host, and then return immediately return control to the invoking shell. Try the following % ssh hyper 'hostname; pwd; date' phys210d@hyper's password: hyper /home/phys210d Fri Aug 31 15:00:14 PDT 2012 % Note that in the above I've used the short version, hyper, of the remote host name, rather than the "fully qualified" form hyper.physics.ubc.ca The is only possible when the two machines belong to the same domain---in this case physics.ubc.ca ------------------------------------------------------------------------- 7) SHELL VARIABLES Recall that in bash there are two types of variables i) Local variables ii) Environment (global) variables ------------------------------------------------------------------------- 7a) Local variables ------------------------------------------------------------------------- Define a few local variables % a=100 % b=foo % c='ls -l' % d='This is a complete sentence.' and then display their values - remember to use the '$' prefix evaluation mechanism % echo $a 100 % echo $b foo % echo $c ls -l % echo $d This is a complete sentence. We can use the value (contents) of c as a command (ls -l) % $c total 12 -rwxr-xr-x 1 phys210d public 39 Aug 23 17:04 cmd* drwxr-xr-x 4 phys210d public 4096 Aug 23 17:04 dir1/ drwxr-xr-x 2 phys210d public 4096 Aug 23 17:04 dir2/ One particularly interesting/useful local variable, PS1 (P-S-one), defines the bash prompt - in these notes I'm pretending that my standard prompt is '% ', but in reality its like yours phys210d@cord ~]$ Display the value of PS1 phys210d@cord ~]$ echo $PS1 [\u@\h \W]$ This output is probably not very intelligible, but if you want the full details see the appropriate section of the online bash manual at http://www.gnu.org/software/bash/manual/bashref.html#Printing-a-Prompt Let's set the prompt to '% ' phys210d@cord ~]$ PS1='% ' % If you want to get it back to the default, source your ~/.bashrc % source ~/.bashrc [phys210d@cord ~]$ ------------------------------------------------------------------------- 2b) Environment (Global) variables ------------------------------------------------------------------------- This type of variable is a little more interesting. First, you can actually get a listing of all currently defined environment (hereafter abbreviated 'env') variables (hereafter abbreviated 'vars', using env % env SSH_AGENT_PID=1931 GPG_AGENT_INFO=/tmp/keyring-Po3r4L/gpg:0:1 TERM=xterm SHELL= XDG_SESSION_COOKIE=6ed81d404965310fcd86745b00000008-1346450839.545947-1440438808 WINDOWID=48246904 GNOME_KEYRING_CONTROL=/tmp/keyring-Po3r4L HISTFILESIZE=1000 USER=phys210d . . . PATH=.:/home/phys210d/bin:/home/phys210/bin:/etc:/usr/etc:/usr/ucb:/bin:/usr/bin:/usr/local/bin:/opt/maple16/bin DESKTOP_SESSION=ubuntu-2d PWD=/home/phys210d . . . HOME=/home/phys210d LANGUAGE=en_CA:en . . . XDG_CURRENT_DESKTOP=Unity COLORTERM=gnome-terminal XAUTHORITY=/home/phys210d/.Xauthority _=/usr/bin/env Note that, by convention, env vars have all-upper-case names. Also note that _ is an env var, and that evaluating it at the shell prompt is the same as executing env % $_ SSH_AGENT_PID=1931 GPG_AGENT_INFO=/tmp/keyring-Po3r4L/gpg:0:1 TERM=xterm . . . XDG_CURRENT_DESKTOP=Unity COLORTERM=gnome-terminal XAUTHORITY=/home/phys210d/.Xauthority _=/usr/bin/env Also note the env vars that we have already mentioned/used in the course, including PATH, HOME, USER and LOGNAME Display the value of one or more specific env vars in one of two ways % echo $HOME /home/phys210d % printenv HOME USER LOGNAME /home/phys210d phys210d phys210d Define some new env vars of our own % FOO='foo' % export FOO 'export' tells bash that FOO is to be an env var, and we can combine the two commands % export BAR='bar' Display the values of FOO and BAR % echo $FOO $BAR foo bar % printenv FOO BAR foo bar Note: The printenv form is preferable, since if we forget to export a var, or mess up the export command, echo will still evaluate the var, but printenv won't find it in the list of environment variables % LOL='lol' % export lol % echo $FOO $LOL foo lol % printenv FOO LOL foo Now comes the interesting part. First, take a look at the value of the 'shell level' env var % printenv SHLVL 1 indicating that this is a "1st-level" shell. Now start up a NEW bash in the original bash % bash % It's hard to tell that anything happened. But check the value of SHLVL % printenv SHLVL 2 so this is a "2nd-level" shell, which makes sense. Now, when we started the new, 2nd-level shell, it inherited all of the env vars (and their values) that were defined in the old shell. So check the values of FOO and BAR % printenv FOO BAR foo bar But the local vars that we defined were NOT inherited, so the following echo command echoes "nothing", i.e. an empty line % echo $a $b $c $d % Return to the old shell % exit exit % Again, hard to tell anything happened except that 'exit' is echoed, and SHLVL is back to 1 % printenv SHLVL 1 You'll get more familiar with the use of shell variables is you want to write bash scripts and functions (optional self-study topic) ------------------------------------------------------------------------- 8) BASH PATTERN MATCHING Recall that pattern matching allows us to select filenames that match a specified pattern (naturally) Let's do our work in a directory that contains a LOT of files (2350 to be exact) cd ~phys210/words You can expect the following ls command to take a bit of time! It will also generate so much output that it will overflow the terminal's buffer, and you won't be able to scroll back to see the first part of the output) % ls A goldeneye prosiness abbas gonadotropic prostatodynia abettal gonococcus protectant ablation goosegirl protestation abortiveness gormaw protococcal . . . Glossotherium promenade zoomantist glume pronaos zoospore glycogenize propagator zygal gnathobasic propiolate zymologic goanna proprietress Godforsaken prorogate Now, let's select various sets of filenames using pattern matching Get a listing of all files whose names ... i) ... begin with z % ls z* zabtie zenithward zoa zoomantist zygal zarabanda zimentwater zonoplacental zoospore zymologic ii) ... have a z in them % ls *z* Aladdinize epitomize nonsympathizer torporize automatize Erethizontidae Nowroze tzaritza azotate femalize ozonoscope uncriticizing bacterize frizziness paronymize unminimized bedazzlingly glycogenize perfectionizement unmunicipalized . . . disequalize monzonitic semihydrobenzoinic zoomantist dizenment muscularize skeletonization zoospore electrocauterization Nazerini squeezing zygal electrolyzer nomadize subzygomatic zymologic emblazer nonanalyzable sucivilized iii) ... end with z % ls *z berkowitz bruzz untz iv) ... are three characters long % ls ??? ask bat icy jib Ker neo nix old Ona Ova ree she vex zoa v) ... begin and end with a vowel % ls [aAeEiIoOuU]*[aAeEiIoOuU] academite asthenia impartance oxyphile aceanthrene athlete impendence ozonoscope acetylsalicylate attune improve Uintatheriidae . . . arthriticine Idiogastra overturnable uranite aschaffite Ilpirra Ovula urubu Asperugo Imerina oxane usucaptable assassinate immotile oxyaphia vi) ... begin and end with a consonant, note consonant = not a vowel % ls [^aAeEiIoOuU]*[^aAeEiIoOuU] babblishly Hippotigris radiator bacchanalianly histiocytic radiology backfall histotrophy raffishly . . . hexadecanoic quinquagenarian zoomantist hidalgoism quintillion zygal hierophantes rabbinistical zymologic Hippeastrum rachiometer vii) ... begin with an a end with a y and have 6 characters % ls a????y archly assify autecy EXERCISE: Try some of your own! ------------------------------------------------------------------------- 9) HISTORY/EVENTS As mentioned in class, the key thing to remember here is that you can type history to see a numbered list of previously typed commands % history should generate quite a lot of output for you by this time, and we'll use that fact to advantage in our work with pipes below. ------------------------------------------------------------------------- 10) STANDARD INPUT and STANDARD OUTPUT Recall that the standard input (stdin) is the default source of input for many Unix/Linux commands, and that standard output (stdout) is the default destination of output for many commands. Both are normally "tied" to the terminal, as best illustrated by executing the cat command with no arguments: since cat reads lines from stdin and writes them to stdout, the net effect is that the command "echoes" every line you type until you terminate the command using ^D (control-D) or ^C (control-C) % cat isn't isn't this this fun fun ? ? ^D Note that I/you only typed four of the lines in the above. Much of the power and utility of the Unix/Linux command line derives from the stdin/stdout concept, plus the mechanisms of stdin/stdout redirection and piping (connecting stdout of one command to stdin of another). ------------------------------------------------------------------------- 11) OUTPUT REDIRECTION and OUTPUT APPEND REDIRECTION This is an important topic, so let's work through a few examples of the two types of output redirection OUTPUT REDIRECTION Go to your home directory, and make a directory called redirect, change to it and verify that you're there % cd % mkdir redirect % cd redirect % pwd /home/phys210d/redirect Now copy all of the (non-hidden) files from ~phys210/redirect to the current directory (i.e. ~/redirect), and get a listing to see what was copied. Note that the second argument to the cp command is '.' (dot). Again, this is shell shorthand for the current directory % cp ~phys210/redirect/* . % ls fireice night pools road snowy The contents of these files are all relatively short poems by Robert Frost. Look at snowy, for example, using more (to control more use the spacebar to go forward a page, 'b' to go back a page and 'q' to quit). % more snowy Whose woods these are I think I know. His house is in the village, though; . . . The woods are lovely, dark and deep, But I have promises to keep, And miles to go before I sleep, And miles to go before I sleep. Now generate a long listing of the directory, but redirect the output of the ls command to a file ls-output % ls -l > ls-output Check the contents of ls-output, using cat (for short files, cat and more are essentially equivalent) % cat ls-output -rw-r--r-- 1 phys210d public 250 Aug 31 2012 fireice -rw-r--r-- 1 phys210d public 0 Aug 31 2012 ls-output -rw-r--r-- 1 phys210d public 614 Aug 31 2012 night -rw-r--r-- 1 phys210d public 539 Aug 31 2012 pools -rw-r--r-- 1 phys210d public 752 Aug 31 2012 road -rw-r--r-- 1 phys210d public 555 Aug 31 2012 snowy Note how ls-output appears in the listing, but has 0 size at the time the listing is generated---i.e. bash creates (or overwrites if noclobber is NOT set) the file to which output is redirected BEFORE it executes the command that generates the output OUTPUT APPEND REDIRECTION Assemble the poems into a single file using cat and the output append redirection operator >> (no intervening whitespace between the two <'s) % cat fireice >> poems % cat night >> poems % cat pools >> poems % cat road >> poems % cat snowy >> poems View the collection of poems % more poems Some say the world will end in fire, Some say in ice. From what I've tasted of desire I hold with those who favor fire. . . . The woods are lovely, dark and deep, But I have promises to keep, And miles to go before I sleep, And miles to go before I sleep. The above example is a bit contrived as the same end could be achieved as follows (note that we first remove poems, and recall that RM is an alias for the bare-bones Unix rm command that doesn't ask for explicit confirmation before removing the file. (forever!!) % RM poems % cat fireice night pools road snowy > poems % more poems Some say the world will end in fire, Some say in ice. From what I've tasted of desire I hold with those who favor fire. . . . The woods are lovely, dark and deep, But I have promises to keep, And miles to go before I sleep, And miles to go before I sleep. ------------------------------------------------------------------------- 12) INPUT REDIRECTION & PIPES Change to your home directory, create the directory numbers, change to numbers, and verify that you are there % cd; mkdir numbers; cd numbers; pwd /home/phys210d/numbers Copy the file ~phys210/numbers/random100 to the working directory (~/numbers) and view the contents of your copy of random100. % cp ~phys210/numbers/random100 . % more random100 15 13 6 2 12 4 6 5 17 8 5 2 18 --More--(12%) random100 contains a list of 100 randomly generated numbers each in the range 1 to 20, with one number per line. Verify that there are indeed 100 numbers (lines) in the file by using the wc (word count) command with the -l (list number of lines only) option % wc -l random100 100 random100 sort is a powerful Unix command that can be used to (surprise, surprise) sort lines from files or standard input according to various criteria. By default (i.e. if executed with NO arguments), sort will i) Read lines from standard input until an end-of-file (EOF) is encountered. Recall that you can generate an EOF at the terminal by typing ^D (control-D) ii) Sort the lines is alphabetical order (case insensitive on the lab machines) iii) Output the sorted lines to standard output Recall that any command that by default reads and writes to stdin and stdout respectively is known as a FILTER. In this exercise we first want to sort the numbers in random100 in ascending numerical order. To do this, we use the -n option (for numeric sort), and redirect the standard input from random100. By default sort outputs to standard output, so we will save the sorted values using output redirection. So let's do the sorting twice, the first with the output going to stdout, the second with output going to the file random100-sorted % sort -n < random100 1 2 2 2 2 . . . 19 20 20 20 20 20 Invoked in this fashion, there's too much information to be seen on the terminal screen. So we have an ideal opportunity to use a pipe; in this case connecting the output of the sort command to the input of more (which also reads from standard input if executed without a filename argument) % sort -n < random100 | more 1 2 2 2 2 2 2 2 3 3 3 3 4 4 4 --More-- So now let's execute the same sort command, but this time redirect the (standard) output to a file random100-sorted % sort -n < random100 > random100-sorted Sure enough random100-sorted contains the numbers in ascending order as was the case when we executed sort -n < random100 | more % more random100-sorted 1 2 2 2 2 2 2 2 3 3 3 3 4 4 4 --More--(11%) QUESTION: Why does more show the percentage of the random100-sorted file that has been displayed when it fills the screen, but does not show a percentage when 'sort -n < random100 | more' is executed? There's another Unix/Linux command, uniq, which like sort, is a filter and which collapses any number of consecutive lines in its input which are identical to a single instance of the line. Thus, we can use sort, uniq and a pipe to generate an ordered list of which integers are contained in random100: % sort -n < random100 | uniq 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 A quick scan indicates that random100 contains at least one instance of each integer between 1 and 20 inclusive, but we can make an even longer pipeline just to make sure! % sort -n < random100 | uniq | wc -l 20 We can get sort to sort in descending order using the -r option % sort -n -r < random100 | uniq 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 And, finally, note that the output from a PIPELINE (not just a single command) can be redirected to a file % sort -n < random100 | uniq | wc -l > nuniq % cat nuniq 20 ------------------------------------------------------------------------- 13) GREP AND REGULAR EXPRESSIONS As mentioned in class, constructing non-trivial regular expressions to use in conjunction with grep constitutes a type of programming in its own right. You should thus expect to have to practice and experiment a bit to get the hang of it. The 3rd and 4th problems of the first homework should definitely provide some practice (and possibly some aggravation, so don't be afraid to ask for hints if you're having trouble!!) Let's first use grep with the file 'poems' that we generated by concatenating the 5 Robert Frost poems in ~/redirect. Just to ensure that everyone is working with the same file, execute the following % cd; mkdir grep; cd grep; cp ~phys210/grep/poems . When you've done this, you should be in the directory ~/grep and should see the following (with phys210d replaced with your login) when you generate a long listing for the the directory % ls -l -rw-r--r-- 1 phys210d public 2710 Sep 4 09:36 poems Now let's use grep to display all the lines in poems that ... i) ... contain the word 'the' (case sensitive) % grep the poems Some say the world will end in fire, I have been one acquainted with the night. I have outwalked the furthest city light. I have looked down the saddest city lane. . . . Whose woods these are I think I know. His house is in the village, though; Between the woods and frozen lake The darkest evening of the year. To ask if there's some mistake. The only other sound's the sweep How many lines are there that contain 'the'? % grep the poems | wc -l 30 And how many lines are there that contain 'the' (case insensitive), so includes The (and conceivably ThE, thE, etc.)? Here we use the -i option, for 'ignore case' % grep -i the poems | wc -l 33 ii) ... start with 'I ' (so that 'I' is in the first column of the line and is followed by at least one space). Here we use the ^ (caret) to "anchor" the search pattern to the beginning of the line, and since the search pattern includes whitespace, we'll need to enclose the regular expression (regexp) in forward quotes. In fact, just to be safe, we'll quote ALL the regexps in the subsequent examples (this is a good habit to develop) % grep '^I ' poems I hold with those who favor fire. I think I know enough of hate I have been one acquainted with the night. I have walked out in rain-and back in rain. I have outwalked the furthest city light. I have looked down the saddest city lane. I have passed by the watchman on his beat I have stood still and stopped the sound of feet I have been one acquainted with the night. I doubted if I should ever come back. I shall be telling this with a sigh I took the one less traveled by, iii) ... start with either 'I' or 'F' (whitespace not necessary after either letter in this case) % grep '^[IF]' poems From what I've tasted of desire I hold with those who favor fire. I think I know enough of hate Is also great I have been one acquainted with the night. I have walked out in rain-and back in rain. I have outwalked the furthest city light. I have looked down the saddest city lane. I have passed by the watchman on his beat I have stood still and stopped the sound of feet I have been one acquainted with the night. From snow that melted only yesterday. In leaves no step had trodden black. I doubted if I should ever come back. I shall be telling this with a sigh I took the one less traveled by, iv) ... start with any upper case letter in the inclusive range T through Z % grep '^[T-Z]' poems To know that for destruction ice When far away an interrupted cry These pools that, though in forests, still reflect The total sky almost without defect, Will like the flowers beside them, soon be gone, The trees that have it in their pent-up buds To darken nature and be summer woods€” To blot out and drink up and sweep away These flowery waters and these watery flowers Two roads diverged in a yellow wood, To where it bent in the undergrowth; Then took the other, as just as fair, Though as for that the passing there Yet knowing how way leads on to way, Two roads diverged in a wood, and I€” Whose woods these are I think I know. To watch his woods fill up with snow. To stop without a farmhouse near The darkest evening of the year. To ask if there's some mistake. The only other sound's the sweep The woods are lovely, dark and deep, v) ... do not start with an upper case letter or a space. This one is a bit tricky since we use ^ for two distinct purposes, one to anchor the search to the beginning of the line, the other to negate the sense of the range A-Z plus space, i.e. [^A-Z ] means 'not in the inclusive range A through Z nor a space. % grep '^[^A-Z ]' poems ... and there's no output, so true to form for this type of poetry, the first word of each line is capitalized (assuming every non-blank line starts in the first column). vi) ... contain only one or more spaces. This one is also a bit tricky. To ensure that there is at least one space the regular expression needs to have an explicit space. We then follow that with a single character range [ ] that can occur ZERO or more times, i.e. [ ]*. Finally, we need to anchor the search to both the beginning of the line using ^ and the end of the line using $ Also note that I deliberately added spaces to empty lines in ~phys210/grep/poems to make this example generate some matches. % grep '^ [ ]*$' poems You should see a lot of blank lines on the screen at this point. How many? Again, pipe the output into wc -l % grep '^ [ ]*$' poems | wc -l 24 OK, two more examples that may help you with the homework vii) ... contain both 'and' and 'the' (case insensitive). We can do this easily by piping two greps together % grep -i 'and' poems | grep -i 'the' I have stood still and stopped the sound of feet And further still at an unearthly height And like the flowers beside them, chill and shiver, These flowery waters and these watery flowers And having perhaps the better claim, And that has made all the difference. Between the woods and frozen lake The woods are lovely, dark and deep, Note carefully that the first grep in the pipeline is supplied with poems as a file argument, but that the second grep is reading from the standard output of the first one, and hence does not need, and should not be given, a file argument viii) ... that are exactly 30 characters long. Here you need to be careful to type precisely 30 periods (.) between the ^ and $ that match the beginning and end of a line, respectively. Recall that '.' matches ANY single character % grep '^..............................$' poems But if it had to perish twice, Somewhere ages and ages hence: = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = Now let's use grep to look for things in our history and list of environment vars. Here, grep will be used in "filter mode", since history or env will supply the input to grep via a pipe. The output, will, of course, be user-dependent Look in the history for grep commands themselves % history | grep grep . . . 1132 mv grep numbers redirect words thedate Save 1227 cd; mkdir grep; cd grep; cp ~phys210/grep/poems . 1229 grep the poems 1230 grep the poems | wc -l 1231 grep -i the poems | wc -l 1232 grep '^I ' poems 1233 grep '^[IF]' poems 1234 grep '^[T-Z]' poems 1235 grep '^[^A-Z ]' poems 1236 grep '^ [ ]*$' poems 1237 grep '^ [ ]*$' poems | wc -l 1238 grep -i 'and' poems | grep -i 'the' 1239 grep '^..............................$' poems 1240 history | grep grep Look in the history for command-lines that begin with cd. Note the construction to match the command-line numbers, as well as the whitespace around the numbers % history | grep '^[ ]*[0-9]*[ ]*cd' . . . 1076 cd 1079 cd; mkdir redirect; cd redirect; pwd 1092 cd ../grep 1129 cd .. 1145 cd 1185 cd ~phys210/words 1197 cd; mkdir redirect; cd redirect; pwd 1213 cd; mkdir numbers; cd numbers; pwd 1227 cd; mkdir grep; cd grep; cp ~phys210/grep/poems . Look for environment vars whose names begin with 'SSH' % env | grep '^SSH' SSH_CLIENT=142.103.234.164 57068 22 SSH_TTY=/dev/pts/24 SSH_CONNECTION=142.103.234.164 57068 142.103.236.100 22 Look for environment vars or values that contain $LOGNAME % env | grep $LOGNAME OLDPWD=/home/phys210d USER=phys210d MAIL=/var/mail/phys210d PATH=.:/home/phys210d/bin:/home/phys210/bin:/etc:/usr/etc:/usr/ucb:/bin:/usr/bin:/usr/local/bin PWD=/home/phys210d/grep HOME=/home/phys210d LOGNAME=phys210d ------------------------------------------------------------------------- 14) ALL THOSE QUOTES We'll keep this section brief since the various quoting mechanisms available, and the distinctions among them, are most relevant in the context of shell programming, which we will not be studying at this time. Thus, we'll simply work through the examples from the notes. FORWARD QUOTES Recall that these quotes inhibit shell evaluation of any and all special characters and/or constructs: % a=100 % echo $a 100 % b=$a % echo $b 100 % b='$a' % echo $b $a Note how the forward quotes stop the shell from replacing $a with its value prior to the assignment to b. DOUBLE QUOTES Recall that these work like single quotes, except that the shell "looks inside" them and evaluates i) any references to the values of shell variables ii) anything within back-quotes (see below) % a=100 % echo $a 100 % string="The value of a is $a" % echo $string The value of a is 100 You should now understand how/why if you defined the alias cdhw1 using % alias cdhw1="cd /phys210/$LOGNAME/hw1" that you see $LOGNAME replaced by your login name when you subsequently ask to see the definition of cdhw1. % alias cdhw1 alias cdhw1='cd /phys210/phys210d/hw1' (If you used your actual account name rather than $LOGNAME in the definition of cdhw1, cdhw2, etc., you might want to try modifying the definitions to use $LOGNAME). BACKWARD QUOTES Recall that these quotes (backticks) provide the useful and powerful mechanism that allows you to capture the output from a command (or, in general, a pipeline) as a string, which can then, for example, be assigned to a shell variable, or supplied as an argument to another command % date Tue Sep 4 09:47:09 PDT 2012 % thedate=`date` % echo $thedate Tue Sep 4 09:47:32 PDT 2012 % which true /bin/true % file `which true` /bin/true: ELF 64-bit LSB executable, x86-64, ... % file `which true` `which false` /bin/true: ELF 32-bit LSB executable ... file `which true` `which false` /bin/true: ELF 32-bit LSB executable ... /bin/false: ELF 32-bit LSB executable ... ----------------------------------------------------------------------------- We've covered a LOT of ground in these notes, and if you can master the material in them you'll be on your way to becoming a Unix/Linux command-line guru! -----------------------------------------------------------------------------