Reducing communication inefficiencies for a flexible programming paradigm

The ACPMAPS system at Fermilab has been upgraded to 50 GF by inserting new CPU modules, based on the Intel 1860. This ten-fold increase in power, utilizing the identical communications backbone, places the system in a different realm: The transfer latency and overheads are now greater, relative to the cost of a floating point operation. We explore the consequences for programs written using CANOPY, which relies on low communications latencies. We present techniques for alleviating the efficiency decrease, by coalescing transfers, without abandoning the considerable advantages of the CANOPY paradigm.