“This storm didn’t even leave one tree with food on it for us to eat,” said an exasperated Esperance, 41, holding a rusty machete in one hand and an overripe breadfruit in the other, not far from one of his farms. “This has taken food out of the mouths of my children.”

Though Haiti was spared a direct hit from Irma and the fallout is nowhere near the magnitude of Matthew’s 546 dead and $2.8 billion in washed-out roads, collapsed bridges and destroyed crops, the frustration and fears for some in its path are no less.

“We didn’t have people who died, but homes and farms were destroyed,” Esperance said. “Just because you don’t see a lot of damages, it doesn’t mean that we haven’t been left deeper in misery.”

The northwest, which was already one of the poorest regions of the poverty-stricken country along with the northeast, was overlooked after Matthew, with attention focused more on the harder-hit Grand’Anse and southwest regions. Northwest farmers, left to fend for themselves, struggled to rebuild, replanting banana, avocado and yam crops to make ends meet.

Then came the worst storm ever recorded in the Atlantic. Irma flooded northern villages from as far west as Môle-Saint-Nicolas to as far east as Ounaminthe on the Haiti-Dominican Republic border. In between, it ripped off roofs, damaged roads and cut off connections between major cities and farming communities.

“If this hurricane had come with a lot of rain, a good amount of people here would be dead,” said Neckson Joseph, 21, a motorcycle taxi driver. “There wasn’t a lot of rain, but there was this strong wind. We had this kitchen outside, covered with aluminum. It didn’t even leave a beam standing.”

Joseph was among scores of storm survivors standing near the banks of a swollen Rivière des Barres in Saint-Louis-du-Nord, a coastal village east of Port-de-Paix, the capital of the northwest. The rising river connects the northwest to the north. Seven communities in the northeast still remained flooded days after Irma’s passage.

Joseph said Irma had felled banana trees, thrown breadfruit and avocados to the ground and swallowed homes along the coast. It also left the coastal villages of Anse-à-Foleur and Côte-de-Fer, and Borgne in the mountains, inundated and cut off from Saint-Louis-du-Nord and Port-de-Paix. The storm also washed out the parts of the road that leads to Jean-Rabel and Môle-Saint-Nicolas in the far west.

“If you really want to see what this storm did, just go walk through the rural outskirts,” Joseph said pointing to the rural inlands on the other side of the Rivière des Barres.

But much closer than that, just west of the Riviere des Barres in Trois Rivières at the entrance of Port-de-Paix, entire banana fields lay in ruin, barely matured plantains strewn on the ground.

“We don’t have any farm,” Camelia Ambroz, 70, said as she and her husband, Charite Almeus, 78, tried to salvage what they could as they walked through soggy, fallen banana branches. “It took everything.”

The couple, who work the land for someone else, said the crops were new, planted shortly after Matthew destroyed the last crop, and had only needed two more months to mature.“When you’re poor and you don’t have anything,” Almeus said, “this is what allows you to live. But what can you do? It’s God’s work and you have to accept it.”

Fritz Jean, an economist who was appointed as prime minister last year but forced to step down after parliament refused to ratify his governance plan for the country, said the vicious cycle of disaster upon disaster in Haiti is making Haitians poorer — and not just farmers.

In the case of farmers, it is leading to “uncertainty in regard to the agricultural production cycle,” he said. “The peasant knows less and less when to sow [and] less and less land becomes available.”

On Friday, when Irma’s red alert — Haiti’s highest level of threat — was finally lifted and a contingent of U.N. Brazilian peacekeepers, making the 200-mile trek from Port-au-Prince, rolled into the northwest, the non-profit group Action Against Hunger warned that a fair amount of the population living in the five regions affected by Irma will feel the impact.

The damages will be extremely difficult for vulnerable, small-scaled farmers in the northwest, the group said.

“These small-scale farmers have no other source of income. If they lose their primary and only source of income, they are left with nothing,” said Mathieu Nabot, Action Against Hunger’s country director. “They have no backup. That is the reason why the impact is strong in the northwest.”

Nabot said that although Irma’s effects are “less dramatic and less visual” than Matthew — which left entire farms under water and coconut trees bent over in the southwest and Grand’Anse — the storm will make life even more difficult. People in the northwest, whether they’re farmers or urban dwellers, are extra vulnerable because of a lack of infrastructure, basic services and economic opportunities in the area.

“Irma is coming after Matthew and Matthew has had a real impact on the ability of people to meet their basic needs in its immediate aftermath,” Nabot said. “They are not yet out of this crisis phase.”

Earlier this week, Haitian Prime Minister Jack Guy Lafontant announced that he had formed a commission to assess Irma’s full impact. The assessment is scheduled to roll out Saturday. The government also tapped Action Against Hunger to coordinate the humanitarian response in collaboration with it and other partners.

Charite Louis, an agronomist who works for Haiti’s food security unit, said he suspects at least 30,000 people will be in need of rapid assistance.

“While it didn’t have the intensity they anticipated, the agriculture sector was hit and the people in that sector are most impacted,” he said. “Some are looking at a total loss… People’s economic means, already fragile, will take a huge hit.”

Long before Matthew left more than 800,000 Haitians in need of emergency food assistance and the El Niño drought phenomenon plunged Haiti into its worst food insecurity crisis in 15 years, the northwest suffered from long dry spells that led to growing food insecurity and deeper poverty despite its potential for fishing and agriculture.

As recently as February, the food insecurity unit classified the northwest as being in an economic and food security crisis. As a result, Nabot said, the focus has to be not just on the emergency response but on supporting farmers over the long term, to help strengthen their economic security and ability to cope with shocks.

“Whatever the crisis, whatever the shock, whatever the [strength] of the hurricane, it will have an impact,” Nabot said. “If not a major or visual one like St. Maarten, it will have an impact on their economic security, that’s for sure.”