I can't comment about the US but I don't think US farmers are poor.
However, in the PH, we don't make much because we plant commodities. The market is only willing to pay a certain price for our goods. In my case, I plant rice which is highly socially sensitive that the government imports rice to keep the price stable. So no upside for us local farmers.
In order to get more value above the commodity rates, we need to differentiate our offerings from the market. Either polish and sell as premium or go organic and label it as such.