Disk quotas

Disk Quotas

Standard quotas for the Math network home directories at this time are: 12 GiB for faculty, graduate students and visitors, and 4 GiB for all other accounts. GiB means 1024³ bytes.

If your account is over this limit, you will be notified upon login and will be given 5 login opportunities to clean up and reduce your disk usage to an under-quota status.

Disk quotas are necessary on a system the size of Math in order to ensure that all users have sufficient disk space available to them.

To avoid further slowing down the file servers, the system currently in place determines each user's disk quota approximately once per day, in the early morning hours. If a user is over quota, then he or she is restricted to five logins until the quota system next determines that the user is within quota. When (or if) that occurs, the status is then cleared.

Thus, login restrictions are imposed or lifted in 24-hour units. This gives a slightly stronger incentive to stay within quotas. It is hoped that users who inadvertently exceed their disk quotas can bring themselves back under quota quickly, and can be fully productive with only a little additional planning to keep from logging in more than five times on that day.

Users who need additional disk space are required to submit a request to request@math.berkeley.edu with a careful explanation of what the excess disk space will be used for.

Notes

There are a few points which often lead to misunderstandings:

• The files in your home directory are compressed/decompressed on the fly while writing/reading. The transparent data compression is enabled in the underlying (ZFS) file system used for home directories. The average compression ratio is about 1.6. This means you may need about 20 GB of space if you want to transfer the data from your network home directory to your own storage device (1 GB = 10⁹ B = 0.93 GiB).

• The disk usage is only determined once per day. Therefore, if one is flagged as being over quota, the login restrictions will remain in effect for the whole day, even if the user immediately deletes files to get back under quota. Conversely, users can go well beyond their quotas with impunity during the day, as long as they reduce their disk usage before the next run of the quota accounting. (This practice is not recommended, because one may forget to delete the files before the end of the day) The reason for running the quota program only once per day is that it involves ascertaining the ownership of every file on a large (home) file system. It is therefore too resource-intensive to be run more often.
• Disk totals are computed by file ownership, not by whether they reside under your home directory. For most users this will not make any difference, but there are circumstances in which this may arise.
• The ls command leaves out files and directories beginning with "." unless the -A option is used.

Quotas on Mail

The quotas described here are different from mail stored on bMail. The campus wide bMail is a part of entirely separate system called bConnected. The bConnected storage was declared "unlimited" on Oct 30, 2014.

(Prior to the conversion to CalMail in 2009 and then to bMail in 2013, the Math department disk quota system applied also to mail boxes used by incoming mail, but that is no longer the case. The output of dqstatus still refers to the separate quota on incoming mail, but that comment is vestigial only.)

Determining Your Disk Usage

There are two programs to aid you in determining your disk usage.

First, there is the program dqstatus (available on the departmental SSH gateway login.math.berkeley.edu). This will print your quota, your disk usage as of the last time the quota system was run, and the number of logins you have left (if you are currently restricted). It may also happen that you have files located in places other than your home directory. These files or directories are also reported by dqstatus.

The other program is du. This will tell you your current disk usage. In particular, the command

du -sk ~

will tell you how much space (in kilobytes) you are using in your home directory. At the beginning of the day this should agree exactly with the output from dqstatus (unless there are files in your home directory that have somebody else as an owner). As the day goes on, however, the numbers will differ since du reflects the current situation while dqstatus is only updated once daily. du is a standard Unix command; it has many other options which are described in the manual page for du.

If you want to know what takes space in your current working directory, use the following command:

du -sk `ls -A` | sort -rn | head

The output will list up to ten largest files or directories including names beginning with "." (specified by the output of ls -A command).