In Ofsted's annual report, Christine Gilbert says England is not world class and disadvantaged children fare worst.
Despite an overall improvement in schools and children's services, the rate of change was too slow, she said.
Ministers say the inspectors have given the "most positive assessment" of the performance of schools.
This is the first time Ofsted has also reported on the new range of areas it inspects and regulates, so it is now covering education, childcare, social care, children's services, adult learning and the skills sector.
Ms Gilbert said too many vulnerable children were being "let down" by the system.
"There's a strong link across every sector between deprivation and poor quality provision," she said.
"In short, if you are poor you are more likely to receive poor services: disadvantage compounds disadvantage."
In education, the annual report said the achievement gap between pupils on free school meals and those who were not had stayed the same for three years.
In 2007, only 21% of children on free school meals achieved the equivalent of five good GCSEs including English and maths, compared with 49% of other pupils.
And while educational achievement was improving overall there was too much variation between areas, and outcomes for disadvantaged pupils broadly had not improved, relative to those from better off homes.
Chief inspector Christine Gilbert said: "To compare favourably with the best in the world, education in England must do better."
The report - which covers school inspections between September 2007 and July 2008 - showed an increase in the number of maintained (state) schools judged to be good or outstanding. Almost two thirds (64%) are in this category.
But almost one in 10 (9%) of secondary schools is said to be inadequate - no change on last year.
There was a slight reduction in the proportion of primary schools said to be inadequate (4%).
The chief inspector criticised achievement levels shown in national tests and examinations.
One in five 11-year-olds was leaving primary school without reaching the level expected of their age group in English and maths, and more than half of England's teenagers were leaving school without five good GCSEs, including English and maths, the report said.
Ms Gilbert summed up: "This report leaves me encouraged by the recognition that so much is going well for so many children, young people and adult learners, but frustrated that there is still too much that is patently inadequate and too many instances where the rate of improvement is unacceptably slow.
"Too many vulnerable children are still being let down by the system and we are failing to learn from the worst cases of abuse."
But Ms Gilbert added it was possible to "buck this trend" and some places were outstanding.
She said: "Typically the provision that really makes a difference is ambitious.
"It does not believe that anyone's past or present circumstances should define their future."
The government said Ofsted had "rightly challenged" it on the effects of poverty and disadvantage on pupils' educational attainment.
But that overall, for schools, this was the "most positive assessment by independent inspectors that we have ever seen of schools' performance".
Schools Minister Jim Knight said: "Teachers should be proud of this assessment against tough inspection standards: 95% of schools are satisfactory or better.
"There are higher proportions of good and outstanding primary and secondary schools. Attendance continues to rise and most pupils behave well.
"There has been significant and sustained improvement in school standards in recent years, with a rise of five percentage points in the proportion of good or outstanding schools since 2005-06."
The government was working to close the achievement gap between poorer children and the more affluent and results were improving for 11-year-olds on free school meals, with the gap narrowing there.
"But we will accelerate this progress, " said Mr Knight.
Christine Blower, Acting General Secretary of the National Union of Teachers (NUT), said: "The fact that the percentage of good and outstanding schools is up is obviously good news and a tribute to teachers and school leaders.
"However, the fact that the chief inspector chooses to link 'poor quality provision' and 'deprivation' is quite frankly not good enough.
"Children from deprived backgrounds are three times more likely to underachieve than their peers from well-off backgrounds. This is a fact, not an excuse."
Dr John Dunford, general secretary of the Association of Schools and College Leaders, said: "The chief inspector's report paints a picture of an improving system against a background of rising expectations placed on schools and colleges.
"It shows schools are using self-review to raise standards of achievement and behaviour and to address the many wider demands now placed on them.
"Secondary schools and colleges are to be congratulated on the rising number graded good or outstanding."