one.world – C# Evaluation

During the summer of 2000, we spent about a week exploring Microsoft's preview release of C#. This page presents our evaluation and is the result of four people programming in C#. The results of our comparison between C# and Java network benchmarks are on a separate page. Microsoft responded to our issues, and we had a follow-up exchange.

The Code

  • One person ported about 18,000 lines (3,900 semi-colons) of well-documented Java code to C# in 2 1/2 days.
  • One person wrote about 1300 lines of code for serializing/deserializing objects as well as querying objects.
  • One person wrote about 650 lines of code for a very simple componetized presentation application.
  • One person wrote about 100 lines of networking code.

Cool Features (that Java doesn't have)

  • Component model
    • Properties
    • Indexers
    • Events
  • Enumerations
  • Attributes
  • Value types (int etc.) are also objects (that is, they are automatically boxed and unboxed)
  • Arrays are also collections (ICollection) --- however, why aren't they also lists (IList)?
  • Delegates, that is, the ability to name an individual method
  • Unsafe code
  • Out and ref parameters to methods, that is multiple results and pass-by-reference --- however, it's unclear whether they are thread-safe
  • Assemblies (collections of related units of code)
  • Guid is a native type
  • Built-in serialization to binary and XML formats (though, documentation seems to be extremely sparse)
  • Built-in code generators for C#, JScript, and Visual Basic and general support for dynamic code generation
  • Security is declarative --- permission checks etc. are declared as attributes on assemblies, classes, and methods and are not embedded inside the code
  • Monitor.Enter() and Monitor.Exit() and reader-writer locks
  • Networking convenience classes such as TCPListener and TCPClient in System.Net.Sockets
  • More than one public class per source file

Issues

  • Exceptions
    • Why aren't thrown exceptions declared?
    • Where is the documentation?
  • Class loading
    • What is the smallest unit of loadable code? Assembly
    • When is code loaded? From where? Typically, on demand from disk --- but seemingly hard to specialize
    • Unclear when code can be unloaded
  • Documentation
    • Where is c#doc? --- It is part of the compiler, use the /doc flag, though it's not nearly as nice as javadoc
    • Lacking in some parts such as exceptions and serialization
  • Abuse of == for string equality (though, only if the variables are declared as strings)
  • namespace is followed by curly brackets and forces a level of indentation
  • Weird keywords
    • base instead of super
    • const and readonly for fields mean pretty much the same
  • Boolean.False is an integer of value 0 and Boolean.True also is an integer
  • Where is the reference queue for weak references?
  • Why is character functionality spread out over System.Char, System.Radix, and System.Globalization.UnicodeCategory (compared with java.lang.Character)?
  • Interfaces are somewhat limited
    • Can't define const fields
    • Can't define enumerations and classes
  • Impossible to directly deserialize objects from the network
  • No inner classes
  • goto instead of labeled continue and break does not preserve intent
  • Integer error codes instead of exceptions (e.g., Socket.Bind())
  • Array.Clone() makes a deep copy, but Hashtable.Clone() makes a shallow copy.
  • How about a common supertype or interface for all numbers?
  • Remove methods for collections should return the removed element
  • Where is the documentation for String.Format?
  • Where is BigInteger?

Performance Numbers

  • Computation & GC
    • Windows 2000 SP1, 750 MHz P III, 128 MB
    • C# technical preview vs. Sun's HotSpot Client VM (build 1.3.0-C)
    • Both running one.eval, a Scheme interpreter
    • TAKeuchi function as a benchmark
      • regular version: deeply recursive
      • continuation-passing version: thousands of continuations
    • 5 runs for each benchmark, measured time excludes startup
    • C# is 6,700 milliseconds on average vs. Java 13,758, which is a factor two slower
  • Serialization
    • Windows 2000, 200 MHz PPro, 64 MB
    • C# technical preview vs. Sun's HotSpot Client VM (build 1.3.0-C)
    • Serialize 10,000 objects into separate buffers, deserialize them again
    • Results are average of 5 runs of each benchmark after 1 warm-up run.
    • Results are first for serialization, then deserialization in milliseconds
      • C# with binary serialization --- 33,425 - 37,391
      • C# with XML serialization --- 7,771 - 8,472
      • Java with binary serialization --- 3,966 - 19,839
      Note that C# with XML serialization generates specialized XML formatters for each class. Above results don't include the generation time, which is 11,049 milliseconds on average.
  • Querying objects
    • Windows 2000, 200 MHz PPro, 64 MB
    • C# technical preview vs. Sun's HotSpot Client VM (build 1.3.0-C)
    • 6 different queries on the fields of an object each 100,000 times
      • 600 lines of code in each language
      • Makes heavy use of reflection in both languages
    • Results are average of 5 runs in milliseconds after 1 warm-up run
    • C# is 9,159 and Java is 105,620
  • Network benchmarking

Trade-offs for C# vs. Java

  • 90% of desktops vs. 20% of programming environments
  • Being pioneers vs. a thriving community
  • Being bug hunters vs. using a mature codebase
  • Being archeologists vs. relying on a large reference library
  • Being evangelists vs. being sheep