Abstract: Picking arrival times of P and S phases is a fundamental and time‐consuming task for the routine processing of seismic data acquired by permanent and temporary networks. A large number of automatic pickers have been developed, but to perform well they often require the tuning of multiple parameters to adapt them to each dataset. Despite the great advance in techniques, some problems remain, such as the difficulty to accurately pick S waves and earthquake recordings with a low signal‐to‐noise ratio. Recently, phase pickers based on deep learning (DL) have shown great potential for event identification and arrival‐time picking. However, the general adoption of these methods for the routine processing of monitoring networks has been held back by factors such as the availability of well‐documented software, computational resources, and a gap in knowledge of these methods. In this study, we evaluate recent available DL pickers for earthquake data, comparing the performance of several neural network architectures. We test the selected pickers using three datasets with different characteristics. We found that the analyzed DL pickers (generalized phase detection, PhaseNet, and EQTransformer) perform well in the three tested cases. They are very efficient at ignoring large‐amplitude transient noise and at picking S waves, a task that is often difficult even for experienced analysts. Nevertheless, the performance of the analyzed DL pickers varies widely in terms of sensitivity and false discovery rate, with some pickers missing a significant percentage of true picks and others producing a large number of false positives. There are also variations in run time between DL pickers, with some of them requiring significant resources to process large datasets. In spite of these drawbacks, we show that DL pickers can be used efficiently to process large seismic datasets and obtain results comparable or better than current standard procedures.