я проблема с флекс-боксом. Может кто-то указать мне верное направление? Возможно даже, что flex-box не может решить эту ситуацию из-за того, как он работает.

Я хочу отображать содержимое в 4 столбцах.

Сделайте так, чтобы содержимое отображалось в столбчатом формате и переносилось к следующему столбцу, когда не хватает места.

display: flex; 
flex-direction: column; 
flex-wrap: wrap; 
height:<##>px

должен позаботиться об этом.

Но здесь все становится сложнее.Что если мне не нужна фиксированная высота, а та, которая увеличивается / уменьшается в зависимости от длины содержимого?

Пусть высота контейнера будет динамической. Как и в случае B, строки выросли до 7 строк из 4 строк в случае A. Содержимое устанавливается так, чтобы заполнить все столбцы настолько, насколько они могут идти.

.other-nearby-cities {
  display: flex;
  flex-direction: column;
  height: 700px;
  flex-wrap: wrap;
  border: solid 1px;
}

<ul class="other-nearby-cities">
  <li><span class="capitalize">Altamonte Springs</span><span> (5)</span></li>
  <li><span class="capitalize">Apopka</span><span> (11)</span></li>
  <li><span class="capitalize">Arcadia</span><span> (11)</span></li>
  <li><span class="capitalize">Auburndale</span><span> (5)</span></li>
  <li><span class="capitalize">Belle Glade</span><span> (10)</span></li>
  <li><span class="capitalize">Boca Raton</span><span> (33)</span></li>
  <li><span class="capitalize">Bonita Springs</span><span> (10)</span></li>
  <li><span class="capitalize">Boynton Beach</span><span> (21)</span></li>
  <li><span class="capitalize">Bradenton</span><span> (46)</span></li>
  <li><span class="capitalize">Brandon</span><span> (16)</span></li>
  <li><span class="capitalize">Brooksville</span><span> (8)</span></li>
  <li><span class="capitalize">Cape Coral</span><span> (17)</span></li>
  <li><span class="capitalize">Clearwater</span><span> (90)</span></li>
  <li><span class="capitalize">Clewiston</span><span> (9)</span></li>
  <li><span class="capitalize">Cocoa</span><span> (18)</span></li>
  <li><span class="capitalize">Crystal River</span><span> (6)</span></li>
  <li><span class="capitalize">Dania Beach</span><span> (7)</span></li>
  <li><span class="capitalize">Davie</span><span> (14)</span></li>
  <li><span class="capitalize">Daytona Beach</span><span> (40)</span></li>
  <li><span class="capitalize">Deerfield Beach</span><span> (15)</span></li>
  <li><span class="capitalize">Deland</span><span> (15)</span></li>
  <li><span class="capitalize">Delray Beach</span><span> (41)</span></li>
  <li><span class="capitalize">Englewood</span><span> (5)</span></li>
  <li><span class="capitalize">Fort Lauderdale</span><span> (132)</span></li>
  <li><span class="capitalize">Fort Myers</span><span> (79)</span></li>
  <li><span class="capitalize">Fort Pierce</span><span> (41)</span></li>
  <li><span class="capitalize">Fort Walton Beach</span><span> (17)</span></li>
  <li><span class="capitalize">Gainesville</span><span> (58)</span></li>
  <li><span class="capitalize">Greenacres</span><span> (9)</span></li>
  <li><span class="capitalize">Hallandale Beach</span><span> (11)</span></li>
  <li><span class="capitalize">Havana</span><span> (6)</span></li>
  <li><span class="capitalize">Hollywood</span><span> (22)</span></li>
  <li><span class="capitalize">Hudson</span><span> (5)</span></li>
  <li><span class="capitalize">Immokalee</span><span> (6)</span></li>
  <li><span class="capitalize">Jacksonville</span><span> (214)</span></li>
  <li><span class="capitalize">Key West</span><span> (9)</span></li>
  <li><span class="capitalize">Kissimmee</span><span> (31)</span></li>
  <li><span class="capitalize">Lake City</span><span> (6)</span></li>
  <li><span class="capitalize">Lake Wales</span><span> (8)</span></li>
  <li><span class="capitalize">Lake Worth</span><span> (26)</span></li>
  <li><span class="capitalize">Lakeland</span><span> (45)</span></li>
  <li><span class="capitalize">Lantana</span><span> (8)</span></li>
  <li><span class="capitalize">Largo</span><span> (17)</span></li>
  <li><span class="capitalize">Lauderdale Lakes</span><span> (15)</span></li>
  <li><span class="capitalize">Lauderhill</span><span> (21)</span></li>
  <li><span class="capitalize">Lecanto</span><span> (7)</span></li>
  <li><span class="capitalize">Leesburg</span><span> (22)</span></li>
  <li><span class="capitalize">Live Oak</span><span> (6)</span></li>
  <li><span class="capitalize">Longwood</span><span> (10)</span></li>
  <li><span class="capitalize">Lutz</span><span> (5)</span></li>
  <li><span class="capitalize">Maitland</span><span> (5)</span></li>
  <li><span class="capitalize">Margate</span><span> (11)</span></li>
  <li><span class="capitalize">Marianna</span><span> (6)</span></li>
  <li><span class="capitalize">Melbourne</span><span> (23)</span></li>
  <li><span class="capitalize">Miami</span><span> (468)</span></li>
  <li><span class="capitalize">Milton</span><span> (6)</span></li>
  <li><span class="capitalize">Miramar</span><span> (23)</span></li>
  <li><span class="capitalize">Naples</span><span> (47)</span></li>
  <li><span class="capitalize">New Port Richey</span><span> (20)</span></li>
  <li><span class="capitalize">New Smyrna</span><span> (5)</span></li>
  <li><span class="capitalize">North Port</span><span> (7)</span></li>
  <li><span class="capitalize">Oakland Park</span><span> (11)</span></li>
  <li><span class="capitalize">Ocala</span><span> (33)</span></li>
  <li><span class="capitalize">Okeechobee</span><span> (9)</span></li>
  <li><span class="capitalize">Orange Park</span><span> (7)</span></li>
  <li><span class="capitalize">Orlando</span><span> (216)</span></li>
  <li><span class="capitalize">Ormond Beach</span><span> (5)</span></li>
  <li><span class="capitalize">Oviedo</span><span> (11)</span></li>
  <li><span class="capitalize">Pahokee</span><span> (5)</span></li>
  <li><span class="capitalize">Palatka</span><span> (13)</span></li>
  <li><span class="capitalize">Palm Bay</span><span> (8)</span></li>
  <li><span class="capitalize">Palm Beach Gardens</span><span> (8)</span></li>
  <li><span class="capitalize">Panama City</span><span> (27)</span></li>
  <li><span class="capitalize">Pembroke Pines</span><span> (24)</span></li>
  <li><span class="capitalize">Pensacola</span><span> (69)</span></li>
  <li><span class="capitalize">Perry</span><span> (5)</span></li>
  <li><span class="capitalize">Plantation</span><span> (11)</span></li>
  <li><span class="capitalize">Pompano Beach</span><span> (32)</span></li>
  <li><span class="capitalize">Port Charlotte</span><span> (15)</span></li>
  <li><span class="capitalize">Port Orange</span><span> (5)</span></li>
  <li><span class="capitalize">Port St Lucie</span><span> (17)</span></li>
  <li><span class="capitalize">Punta Gorda</span><span> (8)</span></li>
  <li><span class="capitalize">Quincy</span><span> (11)</span></li>
  <li><span class="capitalize">Riviera Beach</span><span> (36)</span></li>
  <li><span class="capitalize">Rockledge</span><span> (5)</span></li>
  <li><span class="capitalize">Royal Palm Beach</span><span> (7)</span></li>
  <li><span class="capitalize">Sanford</span><span> (20)</span></li>
  <li><span class="capitalize">Santa Rosa Beach</span><span> (5)</span></li>
  <li><span class="capitalize">Sarasota</span><span> (64)</span></li>
  <li><span class="capitalize">Sebring</span><span> (9)</span></li>
  <li><span class="capitalize">Spring Hill</span><span> (7)</span></li>
  <li><span class="capitalize">St Augustine</span><span> (24)</span></li>
  <li><span class="capitalize">St Petersburg</span><span> (137)</span></li>
  <li><span class="capitalize">Starke</span><span> (5)</span></li>
  <li><span class="capitalize">Stuart</span><span> (23)</span></li>
  <li><span class="capitalize">Sunrise</span><span> (16)</span></li>
  <li><span class="capitalize">Tallahassee</span><span> (131)</span></li>
  <li><span class="capitalize">Tamarac</span><span> (5)</span></li>
  <li><span class="capitalize">Tampa</span><span> (221)</span></li>
  <li><span class="capitalize">Tarpon Springs</span><span> (7)</span></li>
  <li><span class="capitalize">Titusville</span><span> (13)</span></li>
  <li><span class="capitalize">Venice</span><span> (15)</span></li>
  <li><span class="capitalize">Vero Beach</span><span> (31)</span></li>
  <li><span class="capitalize">West Palm Beach</span><span> (135)</span></li>
  <li><span class="capitalize">Winter Garden</span><span> (6)</span></li>
  <li><span class="capitalize">Winter Haven</span><span> (19)</span></li>
  <li><span class="capitalize">Winter Park</span><span> (19)</span></li>
  <li><span class="capitalize">Winter Springs</span><span> (5)</span></li>
</ul>

https://codepen.io/Talamihg/pen/WjMYNd