Elevation can effect the 'micro-climate' due to wind current changes, exposure, etc. I wouldn't worry about it too much... If it really bothers you, and there's not a zone 5 variety for the plant you wish to try, start indoors early, or offer a little protection on colder, windy days and at night for a while. One zone either way hasn't really affected anything I've done - here in IL (700 f.a.s.l.), or at 4000 feet in the mountains of Idaho. My saved seeds have worked perfectly well in both areas, but then I start most everything I can indoors anyway... Just too impatient I guess! Really just depends on how much work you want to do, and how much the particular plants can handle.
SueD