No it is not dumb, it is blessed. To put aside your own desires to visit the poor and disenfranchised is a Godly act, and it means that you have the love of God in you. Such selfLESSness is inspiring and can only lead to future blessings in your life. Look for other opportunities to serve God and he shall richly reward you.