No, you can use any host as long as it's in sync. You can verify this by going to http://www.iotasupport.com/lightwallet.shtml and matching the milestone with those shown in the #botbox channel on the official IOTA Slack channel. If the host that you're currently using becomes de-synced with the network, simply switch hosts ("Tools" > "Edit Node Configuration" > choose a different host from the drop down menu). You may need to close and reopen your wallet for the change to take effect.
If you have any comments or suggestions for "Does it matter which host I use for my light wallet?", please let us know in the comments section below. You may also contact us by submitting a Ticket, or by commenting on the Forum. We would be happy to hear from you.
Thanks for reading.