The official InstallShield 2013 documentation states the following:
The CtrlGetMultCurSel function retrieves the currently selected lines from a multi-selection list box control, that is, a list box control with the LBS_MULTIPLESEL style. (This function does not support extended selection list box controls, that is, list box controls with the LBS_EXTENDEDSEL style.)
Turns out this is not entirely correct. If you set both list box styles (LBS_MULTIPLESEL and LBS_EXTENDEDSEL), then the list box has extended multi-selection style, but CtrlGetMultCurSel() works well with it.