Intel's Anwar Ghuloum writes in a recent blog post that:

Developers should start thinking about tens, hundreds, and thousands of cores now in their algorithmic development and deployment pipeline....

Program for as many cores as possible, even if it is more cores than are currently in shipping [processor] products.

In Unwelcome Advice, Ghuloum goes on to say that:

This starts at a pretty early stage of development; usually, the basic logic of the application should be influenced because it drives the asymptotic parallelism behaviors. Consider a common pattern of optimization we’ve seen in single core tuning: the use of locally adaptive algorithms to heuristically reduce the computation time. By definition, this introduces dependencies in the computation that are beneficial in the single core case but limit parallelism for multi-core. Similar choices are made about libraries and programming languages that optimize for single core performance (or even small-way parallelism), but sacrifice long-term scalability...

Front-loading at least some of this transition is often less costly in the long run and positions them to more competitively reap the benefits of our silicon innovations over time. It’s not quite as simple as this binary choice, but you get the basic idea…program for as many cores as possible, even if it is more cores than are currently in shipping products.

Ghuloum recalls that just a few years ago developers wanting to benefit from parallelism expected that tools, such as parallelizing compilers, would do most of the work. Now, however, developers are beginning to realize that such tools will only go so far, and that basic ideas about code design will have to be re-thought:

Increasingly, we are discussing how to scale performance to core counts that we aren’t yet shipping (but in some cases we’ve hinted heavily that we’re heading in this direction). Dozens, hundreds, and even thousands of cores are not unusual design points around which the conversations meander...

[Taking advantage of that] requires at least some degree of going back to the algorithmic drawing board and rethinking some of the core methods they implement. This also presents the “opportunity” for a major refactoring of their code base, including changes in languages, libraries, and engineering methodologies and conventions they’ve adhered to for (often) most of the their software’s existence.

Ghuloum says that this is actually unpleasant news for many developers and organizations, because it means they have to adapt some well-established practices, including some basic algorithms, and designs to that new reality. At the same time, those quick enough to restructure their code that way will enjoy an advantage over their slower competitors.

To what extent do you consider your application's scalability on many CPU cores?

  function findEmployeesSortedBySalary() : List<Employee> {
    var employees = ...
    return employees.sortBy( \ emp -> emp.Salary )
  }

  function findEmployeesSortedBySalary() : List<Employee> {
    var employees = ...
    return employees.parallelSortBy( \ emp -> emp.Salary )
  }

> function findEmployeesSortedBySalary() : List<Employee>
> e> {
>     var employees = ...
> return employees.parallelSortBy( \ emp -> emp.Salary
> lary )
>   }
>