Skip to content

V
V8
Commit 4ad3fdb1 authored by ricow@chromium.org's avatar ricow@chromium.org
Browse files
Options

Update the ES5 conformance tests to the latest version and fix test 

expectations.

We are currently using a rather old version of the ES5 conformance
tests and there has been a range of bug fixes in the test set. This
change brings us up to date. 

In addition, our current test expectations discards the all chapters
but chapter 15. I have enabled the other chapters, cleaned up the
tests that we no longer fail on, and filed bugs for tests that we fail on.

It seems some of the bugs on the es5 conformance bug-tracker has been
fixed but not marked fixed. I will file bugs for the newly discovered
test bugs.

Review URL: http://codereview.chromium.org/2840023

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@4970 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
parent 7e69e170
Hide whitespace changes
Inline Side-by-side
Showing with 100 additions and 74 deletions
test/es5conform/README
View file @ 4ad3fdb1
...@@ -4,7 +4,7 @@ tests from ...@@ -4,7 +4,7 @@ tests from
https://es5conform.svn.codeplex.com/svn https://es5conform.svn.codeplex.com/svn
in revision 62998 as 'data' in this directory. Using later version in revision 71525 as 'data' in this directory. Using later version
may be possible but the tests are only known to pass (and indeed run) may be possible but the tests are only known to pass (and indeed run)
with that revision. with that revision.
......
test/es5conform/es5conform.status
View file @ 4ad3fdb1
...@@ -29,20 +29,35 @@ prefix es5conform ...@@ -29,20 +29,35 @@ prefix es5conform
def UNIMPLEMENTED = PASS || FAIL def UNIMPLEMENTED = PASS || FAIL
def FAIL_OK = FAIL, OKAY def FAIL_OK = FAIL, OKAY
chapter07: UNIMPLEMENTED # Non UTF8 characters in test files.
chapter08: UNIMPLEMENTED chapter10/10.4/10.4.2/10.4.2-3-c-2-s: FAIL_OK
chapter10: UNIMPLEMENTED chapter10/10.4/10.4.2/10.4.2-3-c-1-s: FAIL_OK
chapter11: UNIMPLEMENTED chapter10/10.4/10.4.2/10.4.2-2-c-1: FAIL_OK
chapter12: UNIMPLEMENTED
chapter13: UNIMPLEMENTED # We do not implement the error chekcs specified in the production rules
chapter14: UNIMPLEMENTED # of 11.1.5 (Object initializer).
chapter15/15.1: UNIMPLEMENTED # We are compatible with Safari and Firefox.
chapter15/15.2/15.2.3/15.2.3.1: UNIMPLEMENTED chapter11/11.1/11.1.5: UNIMPLEMENTED
# Issue 759, see http://code.google.com/p/v8/issues/detail?id=759
chapter11/11.4/11.4.1//11.4.1-4.a-5: FAIL_OK
chapter11/11.4/11.4.1//11.4.1-4.a-7: FAIL_OK
# We do not have a global object called 'global' as required by tests.
chapter15/15.1: FAIL_OK
# NOT IMPLEMENTED: seal
chapter15/15.2/15.2.3/15.2.3.8: UNIMPLEMENTED chapter15/15.2/15.2.3/15.2.3.8: UNIMPLEMENTED
# NOT IMPLEMENTED: freeze
chapter15/15.2/15.2.3/15.2.3.9: UNIMPLEMENTED chapter15/15.2/15.2.3/15.2.3.9: UNIMPLEMENTED
# NOT IMPLEMENTED: preventExtensions
chapter15/15.2/15.2.3/15.2.3.10: UNIMPLEMENTED chapter15/15.2/15.2.3/15.2.3.10: UNIMPLEMENTED
# NOT IMPLEMENTED: isSealed
chapter15/15.2/15.2.3/15.2.3.11: UNIMPLEMENTED chapter15/15.2/15.2.3/15.2.3.11: UNIMPLEMENTED
# NOT IMPLEMENTED: isFrozen
chapter15/15.2/15.2.3/15.2.3.12: UNIMPLEMENTED chapter15/15.2/15.2.3/15.2.3.12: UNIMPLEMENTED
# NOT IMPLEMENTED: isExtensible
chapter15/15.2/15.2.3/15.2.3.13: UNIMPLEMENTED chapter15/15.2/15.2.3/15.2.3.13: UNIMPLEMENTED
# NOT IMPLEMENTED: seal # NOT IMPLEMENTED: seal
...@@ -66,23 +81,23 @@ chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-25: FAIL_OK ...@@ -66,23 +81,23 @@ chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-25: FAIL_OK
# NOT IMPLEMENTED: bind # NOT IMPLEMENTED: bind
chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-38: FAIL_OK chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-38: FAIL_OK
# NaN is writable # NaN is writable.
chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-178: FAIL_OK chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-178: FAIL_OK
# Infinity is writable # Infinity is writable.
chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-179: FAIL_OK chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-179: FAIL_OK
# undefined is writable # undefined is writable.
chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-180: FAIL_OK chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-180: FAIL_OK
# Our Function object has a "arguments" property which is used as a non # Our Function object has an "arguments" property which is used as a
# property in in the test # non-property in the test.
chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-183: FAIL_OK chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-183: FAIL_OK
# Our Function object has a "caller" property which is used as a non # Our Function object has a "caller" property which is used as a
# property in in the test # non-property in in the test.
chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-184: FAIL_OK chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-184: FAIL_OK
# Our function object has a name property which is used as a non # Our function object has a name property which is used as a
# property in the test # non-property in the test.
chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-188: FAIL_OK chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-188: FAIL_OK
# NOT IMPLEMENTED: RegExp.prototype.source # NOT IMPLEMENTED: RegExp.prototype.source
...@@ -102,18 +117,18 @@ chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-215: FAIL_OK ...@@ -102,18 +117,18 @@ chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-215: FAIL_OK
# than those described in the spec - but according to spec they can # than those described in the spec - but according to spec they can
# have additional properties. # have additional properties.
# All compareArray calls in these tests could be exchanged with a # All compareArray calls in these tests could be exchanged with a
# isSubsetOfArray call (I will upload a path to the es5conform site) # isSubsetOfArray call (I will upload a path to the es5conform site).
# SUBSETFAIL # SUBSETFAIL
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-1: FAIL_OK chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-1: FAIL_OK
# SUBSETFAIL + we do not implement all methods on Object # SUBSETFAIL + we do not implement all methods on Object.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-2: FAIL_OK chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-2: FAIL_OK
# SUBSETFAIL # SUBSETFAIL
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-3: FAIL_OK chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-3: FAIL_OK
# SUBSETFAIL + we do not implement Function.prototype.bind # SUBSETFAIL + we do not implement Function.prototype.bind.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-4: FAIL_OK chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-4: FAIL_OK
# SUBSETFAIL # SUBSETFAIL
...@@ -134,28 +149,28 @@ chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-9: FAIL_OK ...@@ -134,28 +149,28 @@ chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-9: FAIL_OK
# SUBSETFAIL # SUBSETFAIL
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-11: FAIL_OK chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-11: FAIL_OK
# We do not implement all methods on RegExp # We do not implement all methods on RegExp.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-13: FAIL_OK chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-13: FAIL_OK
# SUBSETFAIL # SUBSETFAIL
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-14: FAIL_OK chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-14: FAIL_OK
# EvalError.prototype does not have message property # EvalError.prototype does not have message property.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-15: FAIL_OK chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-15: FAIL_OK
# Rangeerror.prototype does not have message property # Rangeerror.prototype does not have message property.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-16: FAIL_OK chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-16: FAIL_OK
# ReferenceError.prototype does not have message property # ReferenceError.prototype does not have message property.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-17: FAIL_OK chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-17: FAIL_OK
# SyntaxError.prototype does not have message property # SyntaxError.prototype does not have message property.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-18: FAIL_OK chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-18: FAIL_OK
# TypeError.prototype does not have message property # TypeError.prototype does not have message property.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-19: FAIL_OK chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-19: FAIL_OK
# URIError.prototype does not have message property # URIError.prototype does not have message property.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-20: FAIL_OK chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-20: FAIL_OK
# SUBSETFAIL # SUBSETFAIL
...@@ -200,68 +215,79 @@ chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-34: FAIL_OK ...@@ -200,68 +215,79 @@ chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-34: FAIL_OK
# SUBSETFAIL # SUBSETFAIL
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-35: FAIL_OK chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-35: FAIL_OK
# NOT IMPLEMENTED: bind on Function.prototype.
chapter15/15.3/15.3.4/15.3.4.5/15.3.4.5-0-1: FAIL_OK
# We fail this because Object.keys returns numbers for element indices # Wrong test - the spec does not say anything about throwing errors
# rather than strings. # on calling Array.prototype.indexOf with undefined as argument.
#chapter15/15.2/15.2.3/15.2.3.14/15.2.3.14-3-3: FAIL_OK chapter15/15.4/15.4.4/15.4.4.14/15.4.4.14-1-1: FAIL_OK
chapter15/15.3: UNIMPLEMENTED # Wrong test - the spec does not say anything about throwing errors
# on calling Array.prototype.indexOf with null as argument.
chapter15/15.4/15.4.4/15.4.4.14/15.4.4.14-1-2: FAIL_OK
chapter15/15.4/15.4.4/15.4.4.14: UNIMPLEMENTED # Wrong assumption - according to spec some returns a Boolean, not a number.
chapter15/15.4/15.4.4/15.4.4.15: UNIMPLEMENTED
chapter15/15.4/15.4.4/15.4.4.20: UNIMPLEMENTED
chapter15/15.4/15.4.4/15.4.4.21: UNIMPLEMENTED
chapter15/15.4/15.4.4/15.4.4.22: UNIMPLEMENTED
# Wrong test - because this is not given as argument to arr.every
# this._15_4_4_16_5_1 evaluates to undefined
chapter15/15.4/15.4.4/15.4.4.16/15.4.4.16-5-1: FAIL_OK
# In test case the element is not appended - it is added in the middle of
# the array
chapter15/15.4/15.4.4/15.4.4.16/15.4.4.16-7-1: FAIL_OK
# We fail because the test assumes that if the reference to array is deleted it
# is not longer traversed
chapter15/15.4/15.4.4/15.4.4.16/15.4.4.16-7-7: FAIL_OK
# if (val>1) in test should be if (val>2)
chapter15/15.4/15.4.4/15.4.4.16/15.4.4.16-8-10: FAIL_OK
# Wrong assumption - according to spec some returns a Boolean, not a number
chapter15/15.4/15.4.4/15.4.4.17/15.4.4.17-4-9: FAIL_OK chapter15/15.4/15.4.4/15.4.4.17/15.4.4.17-4-9: FAIL_OK
# Same as 15.4.4.16-5-1
chapter15/15.4/15.4.4/15.4.4.17/15.4.4.17-5-1: FAIL_OK
# Same as 15.4.4.16-7-1
chapter15/15.4/15.4.4/15.4.4.17/15.4.4.17-7-1: FAIL_OK
# Same as 15.4.4.16-7-7 # Same as 15.4.4.16-7-7
chapter15/15.4/15.4.4/15.4.4.17/15.4.4.17-7-7: FAIL_OK chapter15/15.4/15.4.4/15.4.4.17/15.4.4.17-7-7: FAIL_OK
# Same as 15.4.4.16-10-8 # Same as 15.4.4.16-10-8
chapter15/15.4/15.4.4/15.4.4.17/15.4.4.17-8-10: FAIL_OK chapter15/15.4/15.4.4/15.4.4.17/15.4.4.17-8-10: FAIL_OK
# Same as 15.4.4.16-5-1
chapter15/15.4/15.4.4/15.4.4.18/15.4.4.18-5-1: FAIL_OK
# Same as 15.4.4.16-7-7 # Same as 15.4.4.16-7-7
chapter15/15.4/15.4.4/15.4.4.18/15.4.4.18-7-6: FAIL_OK chapter15/15.4/15.4.4/15.4.4.18/15.4.4.18-7-6: FAIL_OK
# Same as 15.4.4.16-5-1
chapter15/15.4/15.4.4/15.4.4.19/15.4.4.19-5-1: FAIL_OK
# Same as 15.4.4.16-7-7 # Same as 15.4.4.16-7-7
chapter15/15.4/15.4.4/15.4.4.19/15.4.4.19-8-7: FAIL_OK chapter15/15.4/15.4.4/15.4.4.19/15.4.4.19-8-7: FAIL_OK
# Wrong test - uses unitialized variable a in precondition check.
chapter15/15.5: UNIMPLEMENTED chapter15/15.4/15.4.4/15.4.4.19/15.4.4.19-9-3: FAIL_OK
chapter15/15.6: UNIMPLEMENTED
chapter15/15.7: UNIMPLEMENTED # We do not implement Array mapping functions correctly if array
chapter15/15.9: UNIMPLEMENTED # entries are added for nonexistent entries smaller than length by
chapter15/15.10: UNIMPLEMENTED # the callback function.
chapter15/15.12: UNIMPLEMENTED # See http://code.google.com/p/v8/issues/detail?id=755
chapter15/15.4/15.4.4/15.4.4.22/15.4.4.22-9-1: FAIL_OK
# Wrong tests, path in test file is wrong. This will crash the test
# script so we mark it unimplemented.
chapter15/15.4/15.4.4/15.4.4.22/15.4.4.22-9-c-ii-4: UNIMPLEMENTED
chapter15/15.4/15.4.4/15.4.4.22/15.4.4.22-9-c-ii-4-s: UNIMPLEMENTED
# Wrong test - deleting the property on o in callbackfn will
# have no effect on the actual array on which reduceRight is called.
chapter15/15.4/15.4.4/15.4.4.22/15.4.4.22-9-7: FAIL_OK
# We do not implement trim correctly on null and undefined.
chapter15/15.5/15.5.4/15.5.4.20/15.5.4.20-1-1: FAIL_OK
chapter15/15.5/15.5.4/15.5.4.20/15.5.4.20-1-2: FAIL_OK
# We do not correctly recognize \uFEFF as whitespace
chapter15/15.5/15.5.4/15.5.4.20/15.5.4.20-4-10: FAIL_OK
chapter15/15.5/15.5.4/15.5.4.20/15.5.4.20-4-18: FAIL_OK
chapter15/15.5/15.5.4/15.5.4.20/15.5.4.20-4-34: FAIL_OK
# RegExp.prototype is not of type RegExp - we are bug compatible with JSC.
chapter15/15.10/15.10.6/15.10.6: FAIL_OK
# We do not have the properties of a RegExp instance on RegExp.prototype.
# The spec says we should - but we are currently bug compatible with JSC.
chapter15/15.10/15.10.7/15.10.7.1/15.10.7.1-1: FAIL_OK
chapter15/15.10/15.10.7/15.10.7.1/15.10.7.1-2: FAIL_OK
chapter15/15.10/15.10.7/15.10.7.2/15.10.7.2-1: FAIL_OK
chapter15/15.10/15.10.7/15.10.7.2/15.10.7.2-2: FAIL_OK
chapter15/15.10/15.10.7/15.10.7.3/15.10.7.3-1: FAIL_OK
chapter15/15.10/15.10.7/15.10.7.3/15.10.7.3-2: FAIL_OK
chapter15/15.10/15.10.7/15.10.7.4/15.10.7.4-1: FAIL_OK
chapter15/15.10/15.10.7/15.10.7.4/15.10.7.4-2: FAIL_OK
chapter15/15.10/15.10.7/15.10.7.5/15.10.7.5-1: FAIL_OK
chapter15/15.10/15.10.7/15.10.7.5/15.10.7.5-2: FAIL_OK
# JSON.stringify does not correctly truncate the space parameter.
# See http://code.google.com/p/v8/issues/detail?id=753
chapter15/15.12/15.12.3/15.12.3-6-a-2: FAIL_OK
chapter15/15.12/15.12.3/15.12.3-6-b-1: FAIL_OK
[ $arch == mips ] [ $arch == mips ]
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment