The government (federal and state) like to be paid taxes throughout the year, not just as one lump sum on Tax Day (April 15). People who work salaried or hourly jobs have money withheld from their paychecks to meet this requirement. For people like contractors or graduate students receiving a fellowship where taxes are not withheld during the year, taxes need to be paid in (roughly) quarterly installments throughout the year (see below).
If you had income that was not withheld during the quarter, then yes. Although fellowship income is considered "unearned" for tax purposes, it is still considered "income" that must be taxed (unlike, say, a gift). On the other hand, many TA-ships result in "earned income" because this is treated as employment — in this case, as long as you had tax withheld (see on your paystub), then you do not need to pay any quarterly taxes.
Due dates are generally as follows:
If your only income for the year is your stipend, this is pretty easy. Put the stipend amount into an income tax calculator like Smart Asset. Look for the Federal and State (Massachusetts) taxes due. You will pay 1/4 of these totals in taxes each quarter (see links below). If you want to get fancy, observe that there are between 2 - 4 months in each "quarter", and you can pay more or less depending on which quarter it is (because you got paid more or less).
The easiest way to have the money available when it comes time to pay taxes is to calculate how much you will owe overall, then set aside the amount of money you need from each paycheck. Some ways to keep track include budgeting software like Mint, YouNeedABudget, or your own spreadsheet, or making an extra checking or savings account to automatically deposit that amount into.
Did you just start the program after working at a job, or have an income during your undergrad or masters? Did you make money at an internship, co-op, tutoring, an Etsy shop, bitcoin trading, or literally anything else?
If you expect to have more income than the stipend during the year, the math above still applies — but you need to put the total the income you will have into the tax calculator in order for it to find the correct bracket. If you don't consider your other income in the estimation, you will end up under-paying your taxes.
Pro tip: If you had W2 (salaried or hourly) income, see what was withheld for taxes. Usually people have too much withheld over the year — this is why you might get a big return when you do your taxes. If you had other income that was withheld for taxes, you can subtract the respective Federal and State taxes you had withheld from the total Federal and State taxes you will owe for the year when doing your estimated tax calculations. Paying this amount should put you about even when you file your tax return for the year -- you shouldn't get a huge return or have to pay a lot back.