Si uso JDK1.8_40 o más reciente (Oracle u OpenJDK hacen lo mismo), el siguiente código junto con un cambio de tamaño del cuadro de diálogo bloqueará la aplicación (probé Windows 7, x64, JDK de 64 bits)

import javax.swing.JButton;
import javax.swing.JDialog;
import javax.swing.JPanel;
import javax.swing.JWindow;
import javax.swing.SwingUtilities;
import javax.swing.Timer;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;

public class Main {

public static void main(String[] args) {
    SwingUtilities.invokeLater(new Runnable() {
        @Override
        public void run() {
            final JDialog dialog = new JDialog();
            dialog.add(new JPanel());
            dialog.setVisible(true);
            dialog.setBounds(100, 100, 100, 100);

            final JWindow dependentWindow = getjWindow(dialog);
            dependentWindow.setVisible(true);
            dependentWindow.setBounds(100, 100, 100, 100);
            Timer t = new Timer(300, new ActionListener() {
                @Override
                public void actionPerformed(ActionEvent e) {
                    dependentWindow.setVisible(!dependentWindow.isVisible());
                }
            });
            t.start();
        }
    });
}

private static JWindow getjWindow(JDialog dialog) {
    JWindow w = new JWindow(dialog);
    JPanel panel = new JPanel();
    panel.add(new JButton("button"));
    w.add(panel);
    return w;
}
}

No he encontrado otras quejas sobre esto y aún no he publicado un error en el sitio web de Oracle. Una posible solución es cambiar el JWindow a un JDialog no decorado, pero eso viene con otros problemas para mí, por lo que aún no lo cambiaría.

¿Alguien más encontró este problema y encontró una solución?

Se agregó la pila:

WARN 2015-05-04 15:21:21,707 - AWT-EventQueue-0, Id = 17, Priority = 6: RUNNABLE
sun.awt.windows.WWindowPeer.reshapeFrame(Native Method)
sun.awt.windows.WDialogPeer.reshape(Unknown Source)
sun.awt.windows.WComponentPeer.setBounds(Unknown Source)
sun.awt.windows.WWindowPeer.setBounds(Unknown Source)
java.awt.Component.reshapeNativePeer(Unknown Source)
java.awt.Component.reshape(Unknown Source)
java.awt.Window.reshape(Unknown Source)
java.awt.Component.setBounds(Unknown Source)
java.awt.Window.setBounds(Unknown Source)
java.awt.Component.resize(Unknown Source)
java.awt.Component.setSize(Unknown Source)
java.awt.Window.setSize(Unknown Source)

Detalles del problema de Windows (muestra 2 errores):

Problem signature:
Problem Event Name: BEX64
Application Name:   java.exe
Application Version:    8.0.60.13
Application Timestamp:  55404a69
Fault Module Name:  StackHash_08b3
Fault Module Version:   0.0.0.0
Fault Module Timestamp: 00000000
Exception Offset:   0000000300000002
Exception Code: c0000005
Exception Data: 0000000000000008
OS Version: 6.1.7601.2.1.0.256.48
Locale ID:  1033
Additional Information 1:   08b3
Additional Information 2:   08b36dcca93c38acb7c92ef4a729e798
Additional Information 3:   5d68
Additional Information 4:   5d682eddcc7a5d6b5452fc95535d5ac9

segundo:

Problem signature:
Problem Event Name: APPCRASH
Application Name:   java.exe
Application Version:    8.0.60.13
Application Timestamp:  55404a69
Fault Module Name:  StackHash_d693
Fault Module Version:   0.0.0.0
Fault Module Timestamp: 00000000
Exception Code: c000041d
Exception Offset:   0000000300000002
OS Version: 6.1.7601.2.1.0.256.48
Locale ID:  1033
Additional Information 1:   d693
Additional Information 2:   d6933f192f50114566e03a88a59a6417
Additional Information 3:   9096
Additional Information 4:   9096dfe271c183defc2620e74bdaec28