Many important chemical processes—including protein-ligand binding and crystal packing—are dominated by intermolecular interactions. The strength of an interaction between two molecular systems can be rigorously computed with various ab initio quantum chemistry methods, such as symmetry adapted perturbation theory (SAPT), but these computations are often impractically expensive for many systems of interest. Instead, we propose a physically motivated machine learning (ML) framework for approximating intermolecular interactions, which we refer to as AP-Net (short for atomic-pairwise neural network). Unlike other ML models, AP-Net predicts smooth and asymptotically correct potential energy surfaces. We investigate how accuracy is affected by the quantity and type of training data, as well as the chemical interpretability and size-extensivity of predictions. We also explore how AP-Net can be combined with ab initio force field models.