We propose a coronagraphic system with fourth-order null for off-axis segmented telescopes, which is sufficiently insensitive to the telescope pointing errors and finite angular diameter of the host star to enable high-contrast imaging of potentially habitable planets. The inner working angle of the coronagraphic system is close to $1\lambda/D$, and there is no outer limit... The proposed coronagraphic system is made up of a new focal plane mask and an optimized Lyot stop with the second-order null. The new focal plane mask is an extension of the band-limited masks with a phase modulation. We construct a coronagraphic system with fourth-order null by placing two of the new coronagraph systems in succession to be orthogonal to each other. The proposed system is limited to narrow-band usage. The characteristics of the proposed coronagraph system are derived analytically, which includes: (1)the leak of stellar lights due to finite stellar diameter and pointing jitter of a telescope, and (2)the peak throughput. We achieve the performance simulations of this coronagraphic system based on these analytical expressions, considering a monochromatic light of 0.75$\mathrm{\mu}$m and off-axis primary mirror with a diameter of 8.5m. Thanks to the wide working area of the mask, the result shows that terrestrial planets orbiting K and G-dwarfs can be detected under the condition that the telescope pointing jitter is less than $0.01\lambda/D\approx240$as. The proposed coronagraphic system is promising for detection of potentially habitable planets with future space off-axis hexagonally segmented telescopes. (read more)