Dare Obasanjo wrote a good piece (quoting 37Signals) on when to worry about scalability. Read the whole post - his summary is very strong:

If you are a startup, don't waste your time and money worrying about what happens when you have millions of users. Premature optimization is the root of all evil and in certain cases will lead you to being more conservative than you should be when designing features. Remember, even the big guys deal with scalability issues.

On a much smaller scale, I can speak from some experience here. When I first coded up the blog server running this site (back in 2002), I had no idea what kinds of scaling issues I'd hit - heck, at the time, I knew next to nothing about HTTP or XML. Had I spent time trying to solve whatever problems I could have imagined then, it would have been a complete waste. I had to learn from experience. Which is not to say that research and knowledgeable staff won't help; just don't assume that you know the answers up front. As Dare says, this is a problem that even the giants (who have a lot more cash on hand than the rest of us do) run into.