The Bank still needed to see inflation fall further, but last monthʼs drop to 3.4% was "very encouraging and good news," governor Andrew Bailey said.
But he said rate cuts could come before inflation hits its 2% target.
It leaves open the option that the Bank could cut interest rates as early as May, sooner than had been expected.
Mr Bailey sounded far more optimistic on rate cuts than the official statement accompanying the Bankʼs decision to leave the cost of borrowing unchanged for the fifth time in a row.
Most economists had predicted interest rates would start to drop in the summer, especially after inflation fell to its lowest level in two and a half years in February.
"We donʼt have to actually get inflation all the way back to target… to cut rates for instance, what we have to do is be convinced that it is going there," Mr Bailey told the BBCʼs economics editor Faisal Islam who was interviewing him on behalf of UK broadcasters.
"We should act ahead of time in that sense because we have to be forward looking."
He said it was "reasonable" that financial markets were pricing in two or three rate cuts this year, but warned he would not "endorse" that view. "Thatʼs not a prediction from me as to whatʼs going to happen, either on timing or amount, but I am encouraged," he added.
"We do need to see further progress, but I do want to give this message very strongly we have had very encouraging and good news, so I think you know we can say - we are on the way."
The Bank has kept interest rates high in a bid to slow the pace at which consumer prices have been rising.
But eight of the nine Bank rate setters voted to leave rates unchanged in March, with only one voting in favour of a cut. Thursday marked the first time since September 2021 that no one on the Bankʼs committee which decides rates voted for an increase.
What are UK interest rates and when will they fall?
The Bank said it expected inflation to fall slightly below 2% by the summer, but warned that the conflict in the Middle East and disruption to one of the worldʼs busiest shipping lanes in the Red Sea posed "material risks" to prices surging again.
It added its contacts had said the Red Sea situation had delayed shipments by two to three weeks from Asia to Europe and that container costs had risen.
"The goods that are impacted are typically bulky items and components, electrical, clothing and DIY," it said.
ʼImproving outlookʼ
The Bankʼs base rate informs the rates set by High Street banks and lenders. The higher level has meant people are paying more to borrow money for things such as mortgages and credit cards, but savers have also received better returns.
While it is independent from the government, the institution is charged with keeping price growth at, or close to, a target of 2%. It either raises, holds or lowers rates to try to achieve that.
The theory behind increasing rates to tackle inflation is that by making borrowing more expensive, more people will cut back on spending and that leads to demand for goods falling and price rises easing.
But it is a balancing act as high interest rates can harm the economy as businesses hold off on investing in production and jobs.
Following the interest rate increases in 2022, Dharmesh Patadia, a homeowner in Hitchin, has had to deal with a substantial increase in his variable rate mortgage payments.
He is now paying more than three times what he was before the rate rises, an increase of hundreds of pounds.
Mr Patadia says heʼs looking to fix his mortgage rate early once rates begin to fall, despite knowing that rates are likely to drop further and he could potentially get a better deal if he waited, saying that the security of set monthly outgoings is the most important thing for him.
"I just want to have comfort and knowledge that I have a fixed cost and itʼs done."
The UK fell into economic recession at the end of last year when the economy shrank for two consecutive three-month periods, but in their report on Thursday, policymakers suggested the downturn might already be over.
Mr Bailey said there were reasons to believe the UK was "now seeing an end" to the recession.
The Bank said supermarkets had reported growing volumes of sales, consumers were seeking fewer bargains, and seemed "a little more willing to treat themselves, suggesting confidence is improving".
The governmentʼs policies in the Spring Budget were likely to grow the economy by a 0.25% over the "coming years", its report added.
Following Mr Baileyʼs comments on rate cuts, the UKʼs top stock market index soared to its highest level in nearly 11 months. The FTSE 100 jumped by about 2% on Thursday afternoon, its best performance in about six months.