You can use U of T books in the library, but if you want to take them out, you'll need a library card.
Graduate students, faculty members and staff from other Canadian universities are eligible for a Direct Borrower card and all other researchers can apply for a Research Reader card. Undergraduates from other universities and high school students are generally not eligible for borrowing privileges.
Instead of obtaining a U of T library card, you can ask your home library to request a University of Toronto book for you via interlibrary loan. Some libraries charge for this service.