Comments on “ What the Back of the Object Looks Like : 3 D Reconstruction from Line Drawings Without Hidden Lines ”

Abstract— I comment on a paper describing a method for deducing the hidden topology of an object portrayed in a 2D natural line drawing. The principal problem with this paper is that it cannot be considered an advance on (or even an equal of) the state of the art, as the approach it describes makes the same limiting assumptions as approaches proposed ten years ago. There are also important omissions in the review of related work.


INTRODUCTION
IN this paper I comment on the preprint "What the Back of the Object Looks Like: 3D Reconstruction from Line Drawings Without Hidden Lines" (Cao, Liu and Tang).
The aim of the work under discussion is to deduce the topology of that part of an object which cannot be seen in a line drawing portraying that object.Basing such construction on human perception principles, as Cao et al aim to do, is sensible and commendable.However, the paper itself is disappointing.
The basic problem with this paper is that it is not even an advance on the work of Grimstead [2] [3], work which I pointed out to the authors when reviewing an earlier draft of this paper submitted elsewhere.Indeed, Cao et al make the same extremely limiting assumptions.Had their paper been published in 1995 rather than 2005 (the first of many versions was presented to ICCV 2005), it would have equalled the state of the art.As it is, the claim that this is a novel approach to reconstructing a complete 3D object is beside the point.

Previous Work
Section II of Cao et al includes some misconceptions.
Cao et al write: "Face identification from line drawings is not a trivial problem …" This is true for wireframe drawings, but it is not true for natural line drawings.Identifying regions of natural line drawings is trivial.What is non-trivial is the labelling problem: determining which of the lines bounding a region correspond to edges which touch the corresponding face, and which to edges which occlude the corresponding face.
Cao et al mention Marill's MSDA [6], and go on to say that "This idea is followed by many researchers …", listing seven citations, the majority of them to work which specifically does not use Marill's idea, which (as most of them point out) only works well for unusual cases such as the regular polyhedra.
Cao et al write "These reconstruction methods … ( [10] in this paper) … cannot recover complete 3D objects if their hidden lines are not given.Varley and Martin ([11] in this paper) attempted to find the hidden topology of a line drawing representing a manifold polyhedron.However, they had to assume that the 3D geometry of the visible part of the polyhedron has been obtained …" It should be obvious to anyone reading [11] that it represents stage two of a three-stage process.[10] is an update and elaboration of the first stage of this process (there is also an update and elaboration of [11], published in Varley [8] in 2003, which I also pointed out to the authors when reviewing their paper).Taken together, these comprise a solution to the problem which Cao et al investigate.While not a complete solution to this problem, they are a much better approach than that of Cao et al, since the later versions [10][8] allow for the possibilities of non-trihedral vertices and hole loops.

Cao et al continue, "[they had to assume] … the polyhedron has been assigned to one of the
several regular categories".This is a misinterpretation: the polyhedron is assigned to a regularity category, the loosest such category being irregular.[11] and [8] both describe a general case approach for processing irregular objects.

Cao et al conclude "[several regular categories] which can be very difficult to determine
when only the visible part of the object is given."In point of fact, identifying extrusions (the most important special category of object) is trivial.

Assumptions
Assumption 1: The 3D objects are polyhedra with each vertex met by three edges and each edge passed through by two faces, and without through holes.This is the most serious problem with the paper.These restrictions would have been acceptable in 1995, and it appears that the authors are unaware of work done since then.
Furthermore, the authors fail to make it clear whether or not they allow drawings with hole loops or drawings which are not fully graph-connected (the two restrictions, although not exactly equivalent, often amount to the same thing in practice).Not all hole loops represent through holes: some represent pockets, while others represent bosses.None of the test objects used contain bosses or pockets, and none are not fully graph-connected, so the presumption is that Assumption 1 is poorly-worded and the authors also disallow hole loops.
Assumption 2: A line drawing is the parallel or near-parallel projection of the visible edges and vertices of a single polyhedron defined above in a generic view.
The assumption of a generic view is reasonable.It is an assumption made throughout the literature of line drawing interpretation and it is strongly supported by work from the field of perception psychology.
Again, however, the phrasing is unclear.Do the authors assume an accurate parallel or near-parallel projection?If so, this is less reasonable.The aim is to interpret freehand sketches, and these will inevitably contain inaccuracies.

Assumption 3: Every hidden vertex is connected with at least one visible vertex.
This assumption seems to be derived from the method, rather than vice versa.It is admittedly true that the majority of line drawings meet this assumption, but if one were specifying a set of assumptions and then designing an algorithm to meet them, one would not automatically include this assumption.

Cao et al write: "Theorem 1: If a vertex v0 touches a straight line in a line drawing as shown in Fig. 3, then v0 is a broken vertex."
This is true only in the trihedral domain.Even in the extended trihedral domain [7] there are non-occluding T-junctions which correspond to genuine vertices.Once one leaves the trihedral domain, determining which T-junctions are occluding and which are non-occluding is difficult, and indeed whole papers have been written on this problem [9].
By making the assumption that all T-junctions are occluding so early in the proceedings, the authors practically ensure that their work can never be extended past the trihedral domain.
Finally, it is surprising that Cao et al felt it necessary to include a "proof" of this theorem, which is (a) trivial and (b) has been known at least as far back as 1971 [4].
Theorem 2 is a straightforward consequence of Assumption 3, which assumption is challengeable.
Theorems 3 is correct, if not especially enlightening.Lemma 1, used to prove it, is simply the statement that all edges have two ends.
Theorems 4 and 5 appear to be correct, but the soundness of Corollary 1 depends on precise interpretation of loosely-defined terms.For example, it appears that in Fig 2, the three L-junctions along the highlighted path are not on any boundary cycle.
The insight that such sequences of occluding edges represents a single hidden feature of the object is useful, but has been noted before by Grimstead [2].

3D Reconstruction
In Section VI, the authors present an approach to inflating the completed topology into 3D.They also present an entirely separate paper on exactly this problem [5], and I have written an entirely separate critique on that paper.No purpose is served by repeating large chunks of either here.

THE PROPOSED APPROACH
Some of the problems noted so far are problems of presentation.These could be forgiven in a paper which described a good new idea.The idea presented in Section V of the original paper, identifying the topology of hidden vertices directly from an uninflated 2D drawing, is not entirely new (Company et al [1] investigated a similar idea and concluded that it had little potential), but Cao et al have contributed several original refinements and can legitimately claim credit for it.The question remains: is the new idea any good?One could argue that any attempt to choose between various alternatives should take into account as much information as possible, and on that basis inflation should precede any attempt to deduce the hidden topology.This argument is not conclusive in itself-it should be experimental results, not plausible arguments, which decide the matter one way or the other-but it is enough to show that the onus of proof is with those who claim that depth data is not needed, not with those who claim that it is useful.
More to the point, there have been too many dead ends in the past where ideas look plausible when applied to toy objects or limited domains but fail to extend to useful objects or less limited domains.It appears that the approach proposed here is another such.The authors should have done more to demonstrate that their method remains useful outside the domain of toy problems.The authors claim that their algorithm is exponential in the number of hidden edges; it seems to me that it is factorial.They quote a "worst case" of 0.02 seconds.If I am right in believing the algorithm to be factorial, and assuming that the "worst case" is their Fig 16 (seventeen hidden edges), then all we can say about its performance for Fig 3 (thirty-three hidden edges, admittedly close to the limit of what might be drawable in practice) is that it will complete in less than 1.5x10 13 years.
The point may be of minor importance since the situation will be much worse once the trihedral restriction is relaxed and fourth edges are permitted at vertices.

CONCLUSIONS
The principal problem with this paper is that it cannot be considered an advance on, or even an equal of, the state of the art, as the approach it describes makes the same limiting assumptions as approaches proposed ten years ago.There are also important omissions in the review of related work.
In reviewing a previous version of this paper, I concluded, "The authors must justify their work in terms of the advantage of their method as compared with previous work, either in terms of improved results or by virtue of having demonstrably more potential for expansion to general polyhedra and curved objects.At present they do neither."This is just as true of the new version.
By way of illustration, there are comprehensible line drawings which meet the other two assumptions but do not meet assumption 3.One such is the truncated icosahedron, Fig 1, where it is symmetry and familiarity rather than simplicity which are the dominant factors in human perception.

Fig. 3 .
Fig. 3. Fourteen hidden vertices, thirty-three hidden edges If one looks more closely at the set of test examples, one finds that 13 of the 24