"On the one hand we are opening ourselves up to the future. On the other hand, we understand the dangers and will try to balance the two," Prime Minister Benjamin Netanyahu told his cabinet in broadcast remarks.
According to the new policy, which must still be ratified by parliament, people caught smoking marijuana would be fined rather than arrested and prosecuted. Criminal procedures would be launched only against those caught repeatedly with the drug.
Selling and growing marijuana would remain criminal offences in Israel.
"Israel cannot shut its eyes to the changes being made across the world in respect to marijuana consumption and its effects," Justice Minister Ayelet Shaked said in a statement.
In the United States, 28 states have legalized marijuana for medical use and since 2012, several have also approved marijuana for recreational use.