у эти строки кода:

if(my_rank != 0) {
    sprintf(msg, "Hello from %d of %d...", my_rank, comm_sz);
    if(my_rank == 2) {
        sleep(2);
        sprintf(msg, "Hello from %d of %d, I have slept 2 seconds...", my_rank, comm_sz);
    }
    MPI_Send(msg, strlen(msg), MPI_CHAR, 0, 0, MPI_COMM_WORLD);
}
else {
    printf("Hello from the chosen Master %d\n", my_rank);
    for(i = 1; i < comm_sz; i++) {
        MPI_Recv(msg, MAX_STRING, MPI_CHAR, i, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
        printf("%s\n", msg);
    }
}

дать этот результат?

Hello from the chosen Master 0  
Hello from 1 of 5...  
Hello from 2 of 5, I have slept 2 seconds...  
Hello from 3 of 5... have slept 2 seconds...  
Hello from 4 of 5... have slept 2 seconds...

Разве у каждого процесса нет своей копии msg?