In my experience, only the first few months are the hardest time.:-(
I expressed milk before I went to work. Express again during lunch hour. (In washroom, disgusting enough?) In fact, I covered the toilet bowl and then used wet wipe with alcohol to sterilize my hands. Afterwork I expressed milk at once and before bed. As long as I had enough soup and rest, (My son was trained to sleep all night long w/o asking for milk since he was about 8 ~ 10 weeks old.) maintaining milk supply is not difficult.
When my son was 4 months old. He could start eating congee, he did not need as much milk as b4. I did not express milk during lunch hour, just squeeze some milk out from my breast so as to keep stimulation and milk supply. My son had never had milk powder before he was 10 months old. See? It is really possible.
Don't worry! Just remember mother's milk is the best.