In the present paper finite-dimensional dynamical control systems described by semilinear ordinary differential state equations with multiple point delays in control are considered. It is generally assumed, that the values of admissible controls are in a convex and closed cone with vertex at zero. Using so-called generalized open mapping theorem, sufficient conditions for constrained local relative controllability near the origin are formulated and proved. Roughly speaking, it will be proved that under suitable assumptions constrained global relative controllability of a linear associated approximated dynamical system implies constrained local relative controllability near the origin of the original semilinear dynamical system. This is generalization to the constrained controllability case some previous results concerning controllability of linear dynamical systems with multiple point delays in the control and with unconstrained controls. Moreover, necessary and sufficient conditions for constrained global relative controllability of an associated linear dynamical system with multiple point delays in control are discussed. Simple numerical example, which illustrates theoretical considerations is also given. Finally, some remarks and comments on the existing results for controllability of nonlinear dynamical systems are also presented.