Research Paper I
Best viewed at 1024 X 768 resolution and True color (24 bit)

Title:
3-Dimensional Documentation of "Complex Heritage Structures".

(Based on work done in collaboration with Dr. Mudur, Prof. R. K .Joshi, Prof Ranade, Mr. Vivek Punjabi, Bipin Patwardhan, Niranjan Nayak, Alekh Jamsandekar, Shirish Shanbhag, Shrinath Shanbhag, Yogin Masruwala, Dinesh Shikhare and Deepraj Dixit)

Abstract:
Right from the beginning of the first civilizations, documentation of events, religion, culture and significant structures has been of utmost importance for further study and preservation. Various methods of visual communications developed over a period of time ? from ancient techniques like rock cut frescoes, hieroglyphics of Egypt, paintings and carvings in stone to 2-dimensional techniques using paper to today?s digital techniques using multimedia. Today, the computing power at the PC level has increased tremendously, and has allowed the SOHO users to demand better user interface. This need has forced the world of computers not only to switch from 2 dimensional interface to 3 dimensional interface but also increased interactivity. There will soon be a time where a person no longer goes beyond the realms of his own home for experiencing or studying all aspects of any intricate Heritage Structure. Virtual environments of such structures will be easily available and navigable. All the associated information like history, architectural data, archaeological data, culture and also traditional audio will be available at the click of the mouse. There are infinite possibilities of depicting the virtual (different lighting conditions, the structure as it existed (As Was), the present state of the structure (As Is) or future representation of the structure (As It Can Be). Besides marking the beginning of 3-Dimensional documentation of Complex Heritage Structures, this research has extended the limits of human skill (in terms of gathering data and construction of 3D model) and information technology (in terms of hardware limitations and computing power) ? a symbiosis and confluence of arts and information technology.

CONTENTS:
Introduction
The Project: Methodology & Phasing
Digitally narrated history of Fatehpur Sikri
Symbiosis and confluence of various inputs from the creative arts & technical fields
Software
Minimum System requirements
Result and Conclusions
Future Work
References
Acknowledgements

Introduction
This paper describes the advantages and development of methods for 3-Dimensional documentation of "Heritage Structures" for reference and archiving, by taking an example of a Complex Heritage Structure. Till date, most of the important structures of heritage have been documented in the form of orthographic projections. These methods have so far, been the most widely used methods for referencing and archiving. But they still remain non-interactive, very technical, non user-friendly and cumbersome, giving a lot of scope for human errors and missing data. The human vision is binocular and hence the view perceived by a human being is always 3-Dimensional. In this case a virtual 3 dimensional environment becomes the most comfortable environment.

Isometric ViewPaper Model
EARLIER METHODS OF PRESENTATIONS (ISOMETRIC VIEW AND PAPER MODEL OF FATEHPUR SIKRI)
"Fatehpur Sikri" - Marg Publications

Accurate information is required to build such complex models. Even the slightest error is immediately pinpointed and these errors are unpardonable. Such advantages make these virtual environments, the best method for documentation of "Heritage Structures" for reference and archiving.

The Project: Methodology & Phasing
The project was conceived in two parts:

1. "Resurrection of Fatehpur Sikri"
This was the first stage of the project.
A structure of heritage importance, with a sufficient degree of complexity was identified amongst various other alternatives. The palatial complex of Fatehpur Sikri was selected amongst various alternatives for the following reasons:

  1. it posed a very good challenge for modeling its intricate details
  2. most of the 2 dimensional data and archaeological surveys were easily available.
  3. the main structures were well maintained for site study and digital photography.

Plan of Fatehpur Sikri (Orthographic Projection)
Section of Fatehpur Sikri (Orthographic Projection)
Section of Fatehpur Sikri (Orthographic Projection)
ORTHOGRAPHIC PLANS AND SECTIONS OF THE PALACE COMPLEX AND THE MOSQUE COMPLEX
"Mughal Architecture of Fatehpur Sikri" by Edmund Smith and "Fatehpur Sikri" - Marg Publications
Although orthographic projections were the main backbone for this project, due to the 3 dimensional construction of the complex, a lot of missing details (e.g. Height of Buland Darwaza), mismatching dimensions and inaccurate scales and dimensions (e.g. Liwan Cloisters) were encountered. These were cross checked with other references like photographs and on-site measurements. Some of the missing monuments information was recreated from documented history.

This stage was conceived with a challenge in mind, the challenge being depth of detail in terms of physical mesh, which is currently 1.8 million polygons. Detailed plans, elevations, sections and photographs of this world famous heritage building were available through local architectural colleges' libraries. These orthographic projections were the main backbone of the research. But there were some limitations like missing information, incorrect scales and mismatching measurements. Such information was gathered through site survey, available photographs and archaeological data. This provided the necessary data required for the first step towards this challenge.

Diwan-I-Khass Model in ACADPanch Mahal Model in ACADPanch Mahal Model in ACAD
ACCURATE MODELING OF COMPLEX HERITAGE STRUCTURES USING AUTOCAD

The next step was the selection of appropriate software for translating the orthographic projections into 3 dimensional data. A popular and industry standard drafting package, AutoCAD, was selected mainly because of its direct exporting facility to a popular and industry standard visualization package, 3D Studio MAX. The combination worked out to be very powerful for this research, primarily because, highly accurate and precise models could be built using AutoCAD. Meshes were then exported to MAX and manually optimized (removal of unnecessary polygons). Textures were created from available photographs and photographs taken on site. Daylight conditions and possible artificial illuminations were simulated. Site studies were carried out to compare the accuracy of the model. Raytraced animations with audio input and panoramic camera views (giving 360 angle view point) were created to get the feel of the environment.


TEXTURES AS SCANNED, DIGITALLY EDITED AND RETOUCHED BY GRAPHIC ARTIST
Textures were the most important and critical aspect for giving the entire virtual environment, a feeling of reality. There were many limitations involved in using large and untilable maps for the project. A basic requirement of a virtual environment was that the textures had to be to the proportion of 2 to the power "x" and the maximum size was 512 pixels. All the textures had to be made following these guidelines and the texture co-ordinates were used accordingly. Mip maps (same textures replaced by lower resolutions for increased frame rate) were created to avoid wavering effect and aliasing.

A very important issue in this research was to bridge the gap between computing power limitations and level of detail. The first four of the twenty-one structures were built using PC level computers (P-II 300Mhz with 128 MB RAM). Level of detail was satisfactory for still images and animations. But despite employment of clever modeling techniques faster computers were required to complete the whole palace complex. Due to high complexity of the model, computers with specifications like Dual P-II 400Mhz with 512MB RAM and Dual P-II 300Mhz with 384MB RAM were used. These were adequate to achieve a good degree of reality and complete the "Resurrection of Fatehpur Sikri".

2. "Fatehpur Sikri, an epic in Red Sandstone" ? An interactive virtual tour for the PC platform.
This was the second and the final stage of the project.
A walk through engine was developed at the CAD and Graphics Division of National Center for Software Technology, Mumbai, India [4]. "Diwan-I-Khass", a single intricately detailed structure of the Palace complex (approx. 110,000 polygons), was used to test the capability of the walkthrough engine. A characteristic of a detailed model of a Complex Heritage Structure is the concentration of large number of polygons in a small area. This causes the hardware (on PC platform) to take more time for real-time rendering, resulting in an unsatisfactory frame rate. The model was then fragmented into logical and smaller elements for adaptation of culling methods. The mesh was thoroughly optimized by removing double and unwanted faces. Different "level of detail" models were prepared and proximity sensors were used to detect the visibility of meshes. Finally, a satisfactory average frame rate of 13 frames per second was achieved. Textures were mip-mapped and dynamic elements like daylight to night light transitions, rippling water, 3 D surround sound, voice-overs with traditional music and voice recognition techniques for hands free interaction, were added to make the virtual tour a pleasurable experience. Similar techniques were utilized for the entire administrative area and Emperor?s private quarters of "Fatehpur Sikri".
Diwan-I-khass Capitol (Realtime Rendered)Diwan-I-khass Capitol (Wireframe in MAX)
DIWAN-I-KHASS - CAPITOL OVER CENTRAL COLUMN (RAYTRACED IMAGE AND MESH DETAIL IN 3D STUDIO MAX)
A typical problem encountered - "Concentration of large number of polygons in a small area".

The specifications of the model and the application are as follows:
Area under consideration : approx. 600,000 triangles
Diwan-i-Khas capitol: approx. 45,000 triangles
Astrologer's Seat decoration: approx. 32,000 triangles
Textures: approx. 12MB
Textures: approx. 44MB (with shadows and light maps)

Digitally narrated history of Fatehpur Sikri
The following paragraphs explain the history, culture, the complexity of the design and the digital equivalent (in the form of ray-traced and real-time rendered images) of "Fatehpur Sikri".
Compiled from
Indian Architecture, Islamic Period
Fatehpur Sikri, Marg Publication
Mughal Architecture of Fatehpur Sikri by Edmund Smith
Islamic Architecture in India by Satish Grover
The Royal Palaces of India by George Mitchell
The World of Islam by Bernard Lewis
Historic India, Great Ages of Man

The city is divided in 2 parts, the Palace Complex and The Mosque Complex.
The palace complex can be broadly divided in to four parts: Administrative buildings, Emperor's private quarters, and The Harem complex. While the Mosque complex consists of the Buland Darwaza, the Liwan, Salim Chisti's Tomb, Islam Khan's Tomb, Royal Gate, and Zanana Rauza.

All the parts are interconnected by courtyards and cloisters aligned on a mathematically formulated grid pattern, parallel to the great mosque. The planning in form of concentric rings placed the most private areas such as Queen's palaces in the center, surrounded by semi-private areas. The outermost spaces and buildings are meant for service functions and public activities.

Akbar planned a complex on Persian principles, but built and decorated in Indian style. The result is unique, untrammeled by any restriction and exempt from rigid requirements reflecting the mind of the Emperor. Its eclectic but elegant combination of many motifs and adoption of new developments in decorative form is laudable. It showcases the empire's triumph of engineering and structural skills.

The Administrative block is connected to the Emperor's private quarters and to the city. The principal units of the harem are connected to the Emperor's palace by screened corridors and shut off entirely from the administrative areas. A viaduct connects the first floor of Daulat Khana with the Haram Sara; another extends to Hathi pol and beyond to the Hiran Minar. The Mosque complex is approached through the Emperor's gate for the Royalty and by a wide flight of steps for the subjects.

The palace complex is surrounded by several utility structures such as waterworks; gates and community areas; workshops, markets and houses of noblemen, all enclosed within the city walls.

The Administrative area and the Emperor?s Private quarters was selected for the virtual tour. It comprised of the following structures:

Diwan-I-Am: House of Commons
Diwan-I-Am (Real time rendered)
Directly approached from Agra Gate through Naubat Khana, Diwan-I-Am is the place where Akbar presented himself to his subjects and dispensed justice for about 3 hours after sunrise. Elephants brought as a trophy of victory ware tied to the foot of the colonnade opposite the pavilion. Elephants used to crush the condemned to death were also tied here. A colonnade of 111 bays surrounds the court with the Emperor?s platform at the west. The Emperor had direct access to his private apartment, the Daulat Khana from this pavilion.

Diwan-I-Khas: The Jewel House
Diwan-I-khass Capitol (Raytraced Image)Diwan-I-khass Capitol (Raytraced Image)
In this royal chamber for imperial gems and jewels, Akbar sat on the top of the capital of the central column to inspect precious treasures. Abul Fazl records that three separate treasuries were maintained here. One for gold and silver, one for gems and jewels and one for copper. Four entry doors lead to the most astonishing pillar of Mughal architecture. A solitary pillar in the center with 36 brackets supporting a circular platform connected diagonally with 4 stone brides to the galleries running along the upper story.

Ankh Micholi: Treasury
Ankh Michauli and Astrologer's Seat (Ray Traced Image)
This structure was used as an imperial treasury, not as a place to play Ankh Micholi, a game of hide and seek with the ladies of the harem, as the name wrongly suggests. Instead, Akbar bestowed his personal attention upon the management of revenue and expenditure with great care.

Astrologer?s seat: The treasury kiosk
Astrologers Seat (Realtime rendered)
This kiosk is set aside for the Hindu astrologer or Muslim munajjims, highly trained in various systems; each expected to pronounce the Emperor?s prognostications before any major decision was taken. About 3 m. square, this open structure is an extension of the jewel-House platform. An elaborate torana, decorative arches derived from Jain architecture adorn all four openings.

Panch Mahal: Badgir /wind Tower
Panch Mahal (Realtime Rendered)Panch Mahal - nite lights (Realtime Rendered)
Built in five successively receding stories, this open structure comprises of eighty four columns on the ground floor, a number regarded as highly auspicious by Hindus. This number is derived by multiplying seven classically known planets with twelve zodiac signs. The topmost story is a single domed kiosk supported upon for columns making 176 in all. The open structure mitigates heat by deflecting and cooling harsh summer breeze. It also offers breathtaking view of the surroundings.

The Diwan-Khana-I-Khas: Private Palace
The lower walls of this structure are hollowed internally and the openings were closed by sliding stone slabs to store books, read out to the Emperor regularly. This was also used as a dining room. The large room behind this chamber has a platform projecting from its south wall. The Emperor sat here with his legs crossed on scarlet velvet rugs with entrée to the Court standing in line according to their ranks. In the west wall there was a doorway, now closed, giving direct access to the noblemen and clerks from the Daftar Khana to wait upon the Emperor. A window gave onto the quadrangle from near the platform. The Emperor showed himself to His subjects from here daily. A Brahmin interpreter of the great Hindu epic, Mahabharata used to be raised sitting on a charpai, an Indian string bed till he was in level with the Emperor. This is when he instructed the great ruler about the myths and legends of Hinduism.

Khwabgah: Chamber of dreams/ Khilawatkada-I-Khas/ Chitrashala
Khwabgah (Realtime Rendered)Khwabgah with nite lights (Realtime Rendered)
This beautiful chamber on the first floor is Akbar?s private room where ladies from the harem could easily visit him. They also met religious guests and watched court proceedings from here from behind the screens. The Emperor retired here for his short afternoon sleep and for relaxation at night. He held an informal court here with his favorite noblemen such as witty Birbal, Abul Fazl, Nakib Khan and other philosophers and Sufis.

Turkish Sultana?s House: Hujra-I-Anup Talao/ Anup Talao Pavilion
Turkish Sultana's Palace (Realtime Rendered)
This is a place for repose, a zanana residence. It was later connected to the girl?s school at the northwest side and to the Daulat-Khana, emperor?s palace to the southeast with colonnades. Three stone rings are evident 1.98m. above the ground level on the northern side lintels. These were used to fasten canvas tents, as described by Mulla Badouni where important discussions took place.

The Anup Talao: The peerless Pool/ Kapur Talao
Anup Talao (raytraced Image)
At the seat in the middle of Anup Talao, Akbar performed his obligatory prayers five times a day in congregation and granted interviews. Steps give access to water, which came from the northern waterworks; a channel emptied surplus into the Hauz-I-Shirin on the steep southern slope outside the palace.

Abdar khana: Girl?s school
Girl's School and Pachisi Court (Raytraced Image)
There are contradictory stories describing this structure as a girl?s school and/ or as a water and fruit store. One theory suggests that this structure being in the center of the court would not allow girls to use it freely according to Islamic traditions. Other theory describes this as a place where Akbar?s ? water of immortality? was preserved here in the care of a trusted nobleman. The imperial store for the fruits was also established here. The forty courses for the Emperor?s meal was tasted here and served in great style.

The Pachisi Court: Game Court
A part of the pavement of this court is laid out in a cruciform pattern on which the game of Pachisi, also known as choupat, was played with slave girls dressed in different colors, as game pieces while the Emperor sat on a stone seat in the center and dictated the game. Four times a year, the Emperor had sacks; each containing four thousand copper coins piled up here. The money was counted out to court servants, after deduction of profit an ordinary banker would have made.

Symbiosis and confluence of various inputs from the creative arts & technical fields:

  1. Architects were responsible for collection and research of data (archaeological, historical and cultural), maintaining the accuracy of the model and aesthetic evaluations.
  2. Modelers were responsible for accurately interpreting the paper drawings into 3D models, light simulation, mesh optimization for development of virtual environment.
  3. Graphic artists were responsible for preparing digital textures from photographs and retouching existing ones.
  4. Animators produced the key-frames and animation sequences for a guided tour of the complex.
  5. Programmers were responsible for development of walkthrough engine for the PC platform.
  6. Audio specialist was responsible for editing and synchronization of traditional music and voice-overs with the guided tour.
  7. Designers were responsible for development of user-interface and visual communication techniques.

Software Used:

  1. AutoCAD R14 was used for precise interpretation of 2D data and converting it to 3D.
  2. 3D Studio MAX R2.5 was used for texture mapping, lights simulation, rendering, camera settings and panoramic views (360 Degrees View)
  3. Adobe PhotoShop 5.0 was used for digital retouching of textures.
  4. Adobe Premiere 5.0 was used for audio video editing.
  5. Sound Forge 4.5 was used for audio editing.
  6. Visual C++ was used for development of walkthrough engine.
  7. Windows NT 4.0 and Windows ?98 Platforms.

Minimum System requirements:
Pentium(r) III
128MB RAM
8MB AGP Card
Multimedia Kit (speakers, microphone)
CDROM
Windows 98
DirectX 6.1
DirectX 6.0 Media

Result and conclusions:

  1. A fairly accurate 3 D model of a complex Heritage Structure (about 80 %) was achieved using archaeological orthographic projections as reference material.
  2. Construction of a virtual environment could be possible, allowing user to examine or navigate any part of the complex at his own convenient times.
  3. Various dynamic elements like 3d surround sound, day to night transitions, voice-overs for history, culture and other information along with traditional audio pieces and voice recognition techniques could be included to make a complete user-friendly interactive interface.

Future Work:
This method is the advent of a new and interactive form of documenting, referencing and archiving Heritage Structures. The future work involves advanced development of this method.

  1. achievement of better frame rates by using clever culling techniques and modeling techniques for a smooth walkthrough,
  2. development of a powerful walkthrough engine for SOHO users,
  3. incorporation of accurate details by using latest photogrammetric techniques [3],
  4. depiction of "As was", "As is" and "As it could be" transitions for studies specific to architecture, conservation, light simulations,
  5. adaptation of models for the expensive immersive environments (CAVE) [1] and
  6. application to the field of archaeology or conservation using augmented reality [1].

References:
[1] "Virtual and Augmented Reality" by Stefan Mueller, Fraunhofer Institute for Computer Graphics, Germany.
[2] "Three-dimensional representation of the different phases of construction and actual state of conservation of Arenberg Castle using CAAD and Virtual reality applications towards its adequate conservation" by Mario Santana Quintero, Herman Neukarmans, Koenraad Van Balen, Raymond Lemaire Center for Conservation, Belgium.
[3] "Modeling and Rendering Architecture from Photographs" by Paul Debevec, University of California, Berkeley.
[4] "A software Architecture for Virtual Walkthroughs of Complex Heritage Sites" by Dr. S. P. Mudur, National Center for Software Technology, Mumbai, India.
[5] "The Digital Michelangelo Project" by Marc Levoy, Stanford University.

Acknowledgements:
Sir J. J. College of Architecture, Mumbai, India.
Academy of Architecture, Mumbai, India.
L. S. Raheja College of Architecture, Mumbai, India.
Kamala Raheja College of Architecture, Mumbai, India.
National Center for Software Technology, Mumbai, India.
British Council Library, Mumbai, India.
VISIONS, Mumbai, India.
Intel Asia Electronics, Bangalore, India.
Archaeological Survey of India, Aurangabad, India.

and Thanks to:
Mr. Harshad Bhatia, Conservation Architect
Vishwesh Karnik, Architect and Modeler
Mahendra Pore, Graphic Artist
Sanjay Puri, Voice overs

Research by:
Nikhilesh Haval

Design Associate
CAD and Graphics Department
National Center for Software Technology
Bombay, India.

Web version of the project is available at:
http://rohini.ncst.ernet.in/fatehpur

Please visit my webpage for other works:
http://westwood.fortunecity.com/karan/133/index1.htm