Difference between revisions of "File Management"

From UCB Math Wiki
Jump to: navigation, search
(Connecting to the Network: location of computer labs. sftp on Mac/Unix)
m (Using the Macintosh Finder)
 
(11 intermediate revisions by 4 users not shown)
Line 1: Line 1:
This page describes how to manage your files on the math department network, both remotely and locally. File uploading/downloading, directory creation, and setting permissions are among the topics covered belong.
+
This page describes how to manage your files on the math department network, both remotely and locally.
  
===Connecting to the Network===
+
==Connecting to the Network==
  
In order to manage your account, you first have to connect to the network. If you are in Evans, you can log onto one of the departmental computers (such as the ones in 1015 or 838) and instantly be on your math department account. Note that the only computers allowed to be physically connected to the network are the departmental ones. Therefore, if you want to connect from another computer, such as your laptop or home computer, you need to connect remotely. There are two kinds of programs that can connect your computer remotely, SSH and SFTP.
+
In order to manage your account, you first have to connect to the network. If you are in Evans, you can log onto one of the departmental computers (such as thin clients in 744 and public printer rooms) and instantly be on your math department account. Note that the only computers allowed to be physically connected to the network are the departmental ones. Therefore, if you want to connect from another computer, such as your laptop or home computer, you need to connect remotely. There are two kinds of programs that can connect your computer remotely, SSH and SFTP.
  
====SSH====
+
===SSH===
  
 
SSH stands for Secure Shell. For those who have used Telnet before, it is practically the same program except that is has encryption to securely transfer data. The department server no longer accepts connection requests from Telnet clients, so you have to use SSH if you want to connect.
 
SSH stands for Secure Shell. For those who have used Telnet before, it is practically the same program except that is has encryption to securely transfer data. The department server no longer accepts connection requests from Telnet clients, so you have to use SSH if you want to connect.
  
SSH basically is a program that mimics the Terminal windows found on our department's computers. It is useful for editing files, changing file permissions and running basic programs like text editors or Pine email program. With a more elaborate setup, you can actually run programs such as web browsers remotely through SSH (but be warned, it can be an extremely slow process!).
+
SSH basically is a program that mimics the Terminal windows found on our department's computers. It is useful for editing files, changing file permissions and running basic programs like text editors or Pine email program. With a more elaborate setup, you can actually run programs such as web browsers remotely through SSH (but be warned, it can be a slow process!).
  
 
* Windows: you need to download an SSH client. One option is [http://www.chiark.greenend.org.uk/~sgtatham/putty/  Putty]. Then use the SSH client to connect to <code>login.math.berkeley.edu</code>, using your usual departmental username and password.
 
* Windows: you need to download an SSH client. One option is [http://www.chiark.greenend.org.uk/~sgtatham/putty/  Putty]. Then use the SSH client to connect to <code>login.math.berkeley.edu</code>, using your usual departmental username and password.
 
* Mac: you already have SSH installed with your operating system. Just go to Utilities and click on Terminal. In the terminal window, type: <code>ssh -l &lt;username&gt; login.math.berkeley.edu</code> The <code>-l &lt;username&gt;</code> part indicates to SSH that you want to log in with that specific username; if you don't include this text, then SSH will automatically use the account name on your computer (which probably won't be your departmental username).
 
* Mac: you already have SSH installed with your operating system. Just go to Utilities and click on Terminal. In the terminal window, type: <code>ssh -l &lt;username&gt; login.math.berkeley.edu</code> The <code>-l &lt;username&gt;</code> part indicates to SSH that you want to log in with that specific username; if you don't include this text, then SSH will automatically use the account name on your computer (which probably won't be your departmental username).
* Unix: you probably have SSH preinstalled as well. Just open a terminal window and type in the same thing Mac users do.
+
* Linux/Unix: you probably have SSH preinstalled as well. Just open a terminal window and type in the same thing Mac users do.
  
Professor Vojta has the following [http://math.berkeley.edu/computing/vojta/ssh.html reference].
+
Professor Vojta has the following [https://math.berkeley.edu/~vojta/ssh.html reference].
  
====SFTP====
+
===SFTP===
  
 
SFTP stands for Secure File Transfer Protocol, which is a secure version of FTP. As with Telnet, the department server no longer accepts connection requests from FTP, in order to increase network security.
 
SFTP stands for Secure File Transfer Protocol, which is a secure version of FTP. As with Telnet, the department server no longer accepts connection requests from FTP, in order to increase network security.
Line 24: Line 24:
  
 
If you have a command-line <tt>ssh</tt> program on your Mac or Unix computer, you probably have <tt>sftp</tt> as well. A recommendation for Windows is [http://winscp.net/  WinSCP].
 
If you have a command-line <tt>ssh</tt> program on your Mac or Unix computer, you probably have <tt>sftp</tt> as well. A recommendation for Windows is [http://winscp.net/  WinSCP].
 +
 +
===Using the Macintosh Finder===
 +
There are several protocols that will allow you to access and manipulate files on our Unix network via the Macintosh finder. Probably the easiest to use is Netatalk, which unfortunately is not very robustly supported and may be turned off at some point in the future. It already presents various problems: it can be very slow, and sometimes will fail to copy files in either direction for mysterious reasons. At present you can use it from any Macintosh computer, either in your office or from home or another location. After this is done, connecting is easy: just select "Connect to server" in the "Go" menu of the Finder. This will bring up the "Connect to Server" dialog box. Enter "afp://login.math.berkeley.edu". You should then be greeted with another dialog box asking for your password &mdash; your standard Unix password should work. Once the connection goes through, a second dialog will appear, asking you to select a volume to mount. Choose the one with your Unix user name. At this point your Unix home directory should appear as a volume on your Macintosh. Be sure to unmount the volume when you are finished with it.
 +
 +
==Using Unix commands==
 +
 +
Once you've logged into your account, either on one of the department's computers or using [[#SSH|SSH]], you will be presented with a Unix command line.
  
 
===Navigation===
 
===Navigation===
Line 29: Line 36:
 
Every file and directory has a <b>path</b>; if you think of the directories as a tree, then the path is just the path in the tree through all the subdirectories you need to get to that directory or file. For example, your account's home directory is <code>/home/u2/grad/&lt;username&gt;</code>
 
Every file and directory has a <b>path</b>; if you think of the directories as a tree, then the path is just the path in the tree through all the subdirectories you need to get to that directory or file. For example, your account's home directory is <code>/home/u2/grad/&lt;username&gt;</code>
  
* <code>cd &lt;pathname&gt;</code> Change directory to the one given in &lt;pathname&gt;. This command has a very fine distinction between absolute paths and relative paths. If the pathname does not begin with / then it signifies that you are talking about subdirectories of the current one. If it does begin with / then you are specifying the path from the root of the directory tree.
+
* <code>cd &lt;pathname&gt;</code> '''C'''hange '''d'''irectory to the one given in &lt;pathname&gt;. This command has a very fine distinction between absolute paths and relative paths. If the pathname does not begin with / then it signifies that you are talking about subdirectories of the current one. If it does begin with / then you are specifying the path from the root of the directory tree.
* <code>cd ..</code> Go up one level in the heirarchy.
+
 
* <code>pwd</code> Present working directory. This is really useful if you've forgotten what directory you're in.
+
Tip: Use Tab to complete file or directory names.
* <code>ls</code> Lists the files and subdirectories of the present directory.
+
 
* <code>ls -a</code> Lists all the files and subdirectories in the present directory. This includes settings files and other weird junk that usually you don't care about.
+
* <code>cd ..</code> Go up one level in the hierarchy.
 +
* <code>cd</code> With no argument, this command returns you to your home directory.
 +
* <code>pwd</code> Print current directory. This is really useful if you've forgotten what directory you're in.
 +
* <code>ls</code> '''L'''i'''s'''ts the files and subdirectories of the present directory.
 +
* <code>ls -a</code> Lists all the files and subdirectories in the present directory. This includes files which begin with a <code>.</code>, which are usually hidden: mostly settings files and other weird junk that usually you don't care about.
 +
* <code>ls -l</code> Lists files and subdirectories with other information about them, including [[#Permissions|permissions]].
  
 
===Create, Move and Destroy===
 
===Create, Move and Destroy===
Line 39: Line 51:
 
The file server tends to treat directories very differently from files, so if you're going to do some major changes to you account heirarchy it's probably better to use a graphical SFTP program than trying to do this all in a terminal or SSH.
 
The file server tends to treat directories very differently from files, so if you're going to do some major changes to you account heirarchy it's probably better to use a graphical SFTP program than trying to do this all in a terminal or SSH.
  
* <code>mkdir &lt;name&gt;</code> Makes a new directory with the given name.
+
* <code>mkdir &lt;name&gt;</code> '''M'''a'''k'''es a new '''dir'''ectory with the given name.
* <code>cp &lt;filename&gt; &lt;pathname&gt;</code> Creates a copy of file &lt;filename&gt; in the directory given by &lt;pathname&gt;. The same caveat regarding pathnames applies as with <code>cd</code>. This assumes that the file is in the current directory. If you'd like to move a file in a different directory, you have to put the file's path instead of &lt;filename&gt;.
+
* <code>cp &lt;filename&gt; &lt;pathname&gt;</code> Creates a '''c'''o'''p'''y of file &lt;filename&gt; in the directory given by &lt;pathname&gt;. The same caveat regarding pathnames applies as with <code>cd</code>. This assumes that the file is in the current directory. If you'd like to move a file in a different directory, you have to put the file's path instead of &lt;filename&gt;.
* <code>mv &lt;filename&gt; &lt;pathname&gt;</code> Exactly like copy, except it moves the file (so that the file no longer exists in the location it came from.
+
* <code>mv &lt;filename&gt; &lt;pathname&gt;</code> '''M'''o'''v'''es a file to a different location. By specifying a filename at the end of the second path, you can also rename the file.
* <code>rm &lt;filename&gt;</code> Removes (deletes) the file. Be VERY careful if you try to apply this command to a directory--many times the file server won't let you, but if it does and you type things in wrong, the results could be disastrous.
+
* <code>rm &lt;filename&gt;</code> '''R'''e'''m'''oves (deletes) the file. Be careful because unlike moving a file to the trash, once you've typed the <code>rm</code> command there's no way to get the file back. For directories, you have to use:
 +
* <code>rmdir &lt;directoryname&gt;</code> '''R'''e'''m'''oves a '''dir'''ectory. You can only remove a directory if it is empty.
  
 
===Permissions===
 
===Permissions===
  
In order for other people to be able to access your files (in particular, webpages) you have to set the permissions on files and directories appropriately. Every file and folder has many permission variables, but the three that are most relevant are denoted R (read), W (write), and X (execute). By default you have all these permissions and no one else does so if you want to give other people these permissions you have to actually adjust the permissions.
+
Unix has different levels of permissions associated to files which control what you and other people can do. Somewhat confusingly, these are set using a 3 digit number, but the <code>ls -l</code> command displays them using a sequence of letters and dashes. There are many variations, but most of the time the following guidelines work:
 
+
* For things you want other people to see (such as web pages): use 644 (<code>rw-r--r--</code> in <code>ls -l</code>) for files and 755 (<code>rwxr-xr-x</code>) for directories.
A user who has R permission on a file may view the file and R permission on a directory allows you to see the contents. W permission allows the user to change the file, including replacing or deleting it. Only trusted users should be given this permission, and in general it is best to only give yourself this permission. The X permission allows a user to execute a file--this is relevant if you have some sort of web script or other mini-program that external users might want to use.
+
* For things which you don't what other people to read: use 600 (<code>rw-------</code>) for files and 700 (<code>rwx------</code>) for directories.
 
+
The command for setting the permissions on a file is <code>chmod</code>, for example: <code>chmod 644 index.html</code>. For more details, read on.
There are three classes of individuals who may read, write or execute a file; they are Owner (=you), Group (on the math server, this will usually be just you also), and Others. For each set of users, you decide which permissions to give. Each file or directory has a three digit permission code: the first digit corresponds to the permissions for owner, the second digit corresponds to group, and third digit to others.  
 
  
In general, private files should have permissions set to 600 (owner has RW) or 700 (owner has RWX). Public files (such as those needed for your webpage) should be set to 744 in general, or 755 if they are an executable file. Private directories should have permissions set to 700, while public directories should all be set to 755.
+
There are three different permissions and three different classes of users to which these permissions can be assigned. The permissions are read (r), write (w), and execute (x). For files, read and write mean the obvious things. The execute permission gives the ability to run the file as a program. For directories, the interpretations are slightly different: the read permission is the ability to see the list of files in the directory, the write permission allows you to create, move, and delete files in the directory, and the execute permission is necessary to do anything with files in the directory or its subdirectories other than see the list of files in the directory itself. Note that the ability to delete a file depends on having write permission to the directory it's contained in, and not anything with the file itself.
  
To change the file/directory permissions, type <code>chmod &lt;3digitcode&gt; &lt;file/directoryname&gt;</code>
+
The three classes of users which may be given each of these three permissions: owner (i.e. you), group, and others. By default on the math servers, files you create will belong to the <code>grad</code> group, which includes all graduate students. Each of these three classes is assigned some subset of the permissions above. The three-digit code is formed by having one digit for each class (in the order: owner, group, others), and the binary expansion of the digit represents the permissions (read = 4, write = 2, execute = 1). For example, 644 gives read and write permission to the owner and read permission to the group and others.

Latest revision as of 14:01, 20 April 2015

This page describes how to manage your files on the math department network, both remotely and locally.

Connecting to the Network

In order to manage your account, you first have to connect to the network. If you are in Evans, you can log onto one of the departmental computers (such as thin clients in 744 and public printer rooms) and instantly be on your math department account. Note that the only computers allowed to be physically connected to the network are the departmental ones. Therefore, if you want to connect from another computer, such as your laptop or home computer, you need to connect remotely. There are two kinds of programs that can connect your computer remotely, SSH and SFTP.

SSH

SSH stands for Secure Shell. For those who have used Telnet before, it is practically the same program except that is has encryption to securely transfer data. The department server no longer accepts connection requests from Telnet clients, so you have to use SSH if you want to connect.

SSH basically is a program that mimics the Terminal windows found on our department's computers. It is useful for editing files, changing file permissions and running basic programs like text editors or Pine email program. With a more elaborate setup, you can actually run programs such as web browsers remotely through SSH (but be warned, it can be a slow process!).

  • Windows: you need to download an SSH client. One option is Putty. Then use the SSH client to connect to login.math.berkeley.edu, using your usual departmental username and password.
  • Mac: you already have SSH installed with your operating system. Just go to Utilities and click on Terminal. In the terminal window, type: ssh -l <username> login.math.berkeley.edu The -l <username> part indicates to SSH that you want to log in with that specific username; if you don't include this text, then SSH will automatically use the account name on your computer (which probably won't be your departmental username).
  • Linux/Unix: you probably have SSH preinstalled as well. Just open a terminal window and type in the same thing Mac users do.

Professor Vojta has the following reference.

SFTP

SFTP stands for Secure File Transfer Protocol, which is a secure version of FTP. As with Telnet, the department server no longer accepts connection requests from FTP, in order to increase network security.

As the name suggests, this program is primarily used to transfer files between the computer you are on and your account on the department network. Many SFTP clients have a graphical interface which make downloading and uploading very easy, and most also have an easy way of changing file and directory permissions.

If you have a command-line ssh program on your Mac or Unix computer, you probably have sftp as well. A recommendation for Windows is WinSCP.

Using the Macintosh Finder

There are several protocols that will allow you to access and manipulate files on our Unix network via the Macintosh finder. Probably the easiest to use is Netatalk, which unfortunately is not very robustly supported and may be turned off at some point in the future. It already presents various problems: it can be very slow, and sometimes will fail to copy files in either direction for mysterious reasons. At present you can use it from any Macintosh computer, either in your office or from home or another location. After this is done, connecting is easy: just select "Connect to server" in the "Go" menu of the Finder. This will bring up the "Connect to Server" dialog box. Enter "afp://login.math.berkeley.edu". You should then be greeted with another dialog box asking for your password — your standard Unix password should work. Once the connection goes through, a second dialog will appear, asking you to select a volume to mount. Choose the one with your Unix user name. At this point your Unix home directory should appear as a volume on your Macintosh. Be sure to unmount the volume when you are finished with it.

Using Unix commands

Once you've logged into your account, either on one of the department's computers or using SSH, you will be presented with a Unix command line.

Navigation

Every file and directory has a path; if you think of the directories as a tree, then the path is just the path in the tree through all the subdirectories you need to get to that directory or file. For example, your account's home directory is /home/u2/grad/<username>

  • cd <pathname> Change directory to the one given in <pathname>. This command has a very fine distinction between absolute paths and relative paths. If the pathname does not begin with / then it signifies that you are talking about subdirectories of the current one. If it does begin with / then you are specifying the path from the root of the directory tree.

Tip: Use Tab to complete file or directory names.

  • cd .. Go up one level in the hierarchy.
  • cd With no argument, this command returns you to your home directory.
  • pwd Print current directory. This is really useful if you've forgotten what directory you're in.
  • ls Lists the files and subdirectories of the present directory.
  • ls -a Lists all the files and subdirectories in the present directory. This includes files which begin with a ., which are usually hidden: mostly settings files and other weird junk that usually you don't care about.
  • ls -l Lists files and subdirectories with other information about them, including permissions.

Create, Move and Destroy

The file server tends to treat directories very differently from files, so if you're going to do some major changes to you account heirarchy it's probably better to use a graphical SFTP program than trying to do this all in a terminal or SSH.

  • mkdir <name> Makes a new directory with the given name.
  • cp <filename> <pathname> Creates a copy of file <filename> in the directory given by <pathname>. The same caveat regarding pathnames applies as with cd. This assumes that the file is in the current directory. If you'd like to move a file in a different directory, you have to put the file's path instead of <filename>.
  • mv <filename> <pathname> Moves a file to a different location. By specifying a filename at the end of the second path, you can also rename the file.
  • rm <filename> Removes (deletes) the file. Be careful because unlike moving a file to the trash, once you've typed the rm command there's no way to get the file back. For directories, you have to use:
  • rmdir <directoryname> Removes a directory. You can only remove a directory if it is empty.

Permissions

Unix has different levels of permissions associated to files which control what you and other people can do. Somewhat confusingly, these are set using a 3 digit number, but the ls -l command displays them using a sequence of letters and dashes. There are many variations, but most of the time the following guidelines work:

  • For things you want other people to see (such as web pages): use 644 (rw-r--r-- in ls -l) for files and 755 (rwxr-xr-x) for directories.
  • For things which you don't what other people to read: use 600 (rw-------) for files and 700 (rwx------) for directories.

The command for setting the permissions on a file is chmod, for example: chmod 644 index.html. For more details, read on.

There are three different permissions and three different classes of users to which these permissions can be assigned. The permissions are read (r), write (w), and execute (x). For files, read and write mean the obvious things. The execute permission gives the ability to run the file as a program. For directories, the interpretations are slightly different: the read permission is the ability to see the list of files in the directory, the write permission allows you to create, move, and delete files in the directory, and the execute permission is necessary to do anything with files in the directory or its subdirectories other than see the list of files in the directory itself. Note that the ability to delete a file depends on having write permission to the directory it's contained in, and not anything with the file itself.

The three classes of users which may be given each of these three permissions: owner (i.e. you), group, and others. By default on the math servers, files you create will belong to the grad group, which includes all graduate students. Each of these three classes is assigned some subset of the permissions above. The three-digit code is formed by having one digit for each class (in the order: owner, group, others), and the binary expansion of the digit represents the permissions (read = 4, write = 2, execute = 1). For example, 644 gives read and write permission to the owner and read permission to the group and others.