The cost of uncompressing (decoding) data can be prohibitive in certain real-time applications, for example when predicting using compressed deep learning models. In many scenarios, it is acceptable to sacrifice to some extent on compression in the interest of fast decoding. In this work, we are interested in finding the prefix tree having the best decode time under the constraint that the code length does not exceed a certain threshold for a natural class of algorithms under the hierarchical memory model. We present an efficient optimal algorithm for this problem based on a dynamic program that capitalizes on an interesting structure of the optimal solution. © 2020 IEEE.