LaTeX basics
LaTeX is the standard method of typesetting mathematics. It comes on all the department's computers. See here for instructions on installing it on your own computer. LaTeX's input can be edited in any text editor, but you may wish to use a specialized text editor.
Learning LaTeX
"An Introduction to LaTeX" and "Getting Started with LaTeX" are good introductions.
"The Not-so-short introduction to LaTeX2e" is very complete, and also written in a very easy to understand fashion. It's probably more useful as a reference than anything else—I don't recommend reading it cover-to-cover.
"Using the exam document class" is a very readable introduction to a very useful document class.
Once you've learned the basics, a good idea is to get a LaTeX file from a more experienced user so that you can see their tricks in action, and just cut and paste. Even experienced LaTeX users do this from their own papers from time to time. For example, you can get the source of any paper on the arXiv, which should lead to a fairly inexhaustible source of LaTeX tricks beyond the standard tutorials.
If you want to use the exam package, you could use this practice final.
Diagrams
The most popular package for drawing diagrams is XYpic. Drawing roughly rectilinear commutative diagrams is relatively easy (see the xyguide), but in fact, there is a lot more you can do with this package if you're willing to go through the steeper learning curve and learn xypic proper. Aaron Lauda has a lot of useful examples in his tutorial.
Other options include the amscd and diagrams packages which are less flexible, but easier for simpler commutative diagrams.
Bibliographies
Learn how to use BibTeX. I promise you won't regret it later. Remember that you can get BibTeX citations from from MathSciNet or the Front. Somewhat obnoxiously, the standard BibTeX styles won't produce a citation to the arXiv, so you'll want an special BibTeX style. I use halpha.bst.
If ever want access to the LaTeX version of the bibliography that BibTeX has created, it's in the file *.bbl. If you submit a paper with BibTeX'ed bibliography to the arXiv, you should include the *.bbl file, NOT the *.bib.
