Both days are certainly important. Without being born there is obviously no person to talk about! However, the day of leaving this world – often called “Yahrtzeit” – is a more important day in terms of remembering the person.
Why is that? The Torah teaches that the significance of the Yahrzeit is that a person leave's an imprint on the world. When a person comes into the world it is unclear what kind of imprint they will leave which means that there is an uncertainly as to what will be achieved. This uncertainty is only decided at the end of one’s life in this world, when it becomes clear what righteous and good deeds the person has accomplished in his lifetime.