Skip to main content Accessibility help
×
Home

Testing noninterference, quickly

  • CĂTĂLIN HRIŢCU (a1), LEONIDAS LAMPROPOULOS (a2), ANTAL SPECTOR-ZABUSKY (a2), ARTHUR AZEVEDO DE AMORIM (a2), MAXIME DÉNÈS (a3), JOHN HUGHES (a4), BENJAMIN C. PIERCE (a2) and DIMITRIOS VYTINIOTIS (a5)...

Abstract

Information-flow control mechanisms are difficult both to design and to prove correct. To reduce the time wasted on doomed proof attempts due to broken definitions, we advocate modern random-testing techniques for finding counterexamples during the design process. We show how to use QuickCheck, a property-based random-testing tool, to guide the design of increasingly complex information-flow abstract machines, leading up to a sophisticated register machine with a novel and highly permissive flow-sensitive dynamic enforcement mechanism that is sound in the presence of first-class public labels. We find that both sophisticated strategies for generating well-distributed random programs and readily falsifiable formulations of noninterference properties are critically important for efficient testing. We propose several approaches and evaluate their effectiveness on a collection of injected bugs of varying subtlety. We also present an effective technique for shrinking large counterexamples to minimal, easily comprehensible ones. Taken together, our best methods enable us to quickly and automatically generate simple counterexamples for more than 45 bugs. Moreover, we show how testing guides the discovery of the sophisticated invariants needed for the noninterference proof of our most complex machine.

Copyright

References

Hide All
Antoy, S. (2000) A needed narrowing strategy. J. ACM 47 (4), 776822.
Austin, T. H. & Flanagan, C. (2009) Efficient purely-dynamic information flow analysis. In Proceedings of 4th ACM SIGPLAN Workshop on Programming Languages and Analysis for Security, PLAS'09. ACM, pp. 113–124.
Austin, T. H. & Flanagan, C. (2010) Permissive dynamic information flow analysis. In Proceedings of 5th ACM SIGPLAN Workshop on Programming Languages and Analysis for Security, PLAS'10. ACM, pp. 3:1–3:12.
Azevedo de Amorim, A., Collins, N., DeHon, A., Demange, D., Hriţcu, C., Pichardie, D., Pierce, B. C., Pollack, R. & Tolmach, A. (January 2014) A verified information-flow architecture. In Proceedings of 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL'14. ACM, pp. 165–178.
Azevedo de Amorim, A., Dénès, M., Giannarakis, N., Hriţcu, C., Pierce, B. C., Spector-Zabusky, A. & Tolmach, A. (2015) Micro-policies: Formally verified, tag-based security monitors. In Proceedings of 36th IEEE Symposium on Security and Privacy, SP'15. IEEE, pp. 813–830.
Balliu, M., Dam, M. & Guernic, G. L. (2012) Encover: Symbolic exploration for information flow security. In Proceedings of 25th IEEE Computer Security Foundations Symposium, CSF'12. IEEE, pp. 30–44.
Banerjee, A. & Naumann, D. A. (2005) Stack-based access control and secure information flow. J. Funct. Program. 15 (2), 131177.
Barthe, G., Crespo, J. M. & Kunz, C. (2011a) Relational verification using product programs. In Proceedings of 17th International Symposium on Formal Methods, FM'11, Lecture Notes in Computer Science, vol. 6664. Springer, pp. 200–214.
Barthe, G., D'Argenio, P. R. & Rezk, T. (2011b) Secure information flow by self-composition. Math. Struct. Comput. Sci. 21 (6), 12071252.
Benton, N. (2004) Simple relational correctness proofs for static analyses and program transformations. In Proceedings of 31st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL'04. ACM, pp. 14–25.
Berghofer, S. & Nipkow, T. (2004) Random testing in Isabelle/HOL. In Proceedings of 2nd International Conference on Software Engineering and Formal Methods, SEFM'04. IEEE CS, pp. 230–239.
Bichhawat, A., Rajani, V., Garg, D. & Hammer, C. (2014a) Information flow control in WebKit's JavaScript bytecode. In Proceedings of 3rd International Conference on Principles of Security and Trust, POST'14, Lecture Notes in Computer Science, vol. 8414. Springer, pp. 159–178.
Bichhawat, A., Rajani, V., Garg, D. & Hammer, C. (2014b) Generalizing permissive-upgrade in dynamic information flow analysis. In Proceedings of 9th Workshop on Programming Languages and Analysis for Security, PLAS'14. ACM, pp. 15–24.
Birgisson, A., Hedin, D. & Sabelfeld, A. (2012) Boosting the permissiveness of dynamic information-flow tracking by testing. In Proceedings of 17th European Symposium on Research in Computer Security, ESORICS'12, Lecture Notes in Computer Science, vol. 7459. Springer, pp. 55–72.
Buiras, P., Stefan, D., & Russo, A. (2014) On dynamic flow-sensitive floating-label systems. In Proceedings of 27th IEEE Computer Security Foundations Symposium, CSF'14. IEEE, pp. 65–79.
Bulwahn, L. (2012a) The new Quickcheck for Isabelle - random, exhaustive and symbolic testing under one roof. In Proceedings of 2nd International Conference on Certified Programs and Proofs, CPP'12, Lecture Notes in Computer Science, vol. 7679. Springer, pp. 92–108.
Bulwahn, L. (2012b) Smart testing of functional programs in Isabelle. In Proceedings of 18th International Conference on Logic for Programming, Artificial Intelligence, and Reasoning, LPAR'12, Lecture Notes in Computer Science, vol. 7180, Springer, pp. 153–167.
Burnim, J. & Sen, K. (2008) Heuristics for scalable dynamic test generation. In Proceedings of 23rd IEEE/ACM International Conference on Automated Software Engineering, ASE'08, IEEE Computer Society, pp. 443–446.
Cadar, C., Dunbar, D. & Engler, D. (2008) KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of 8th USENIX Conference on Operating Systems Design and Implementation, OSDI'08. USENIX Association, pp. 209–224.
Cadar, C., Ganesh, V., Pawlowski, P. M., Dill, D. L. & Engler, D. R. (2006) EXE: Automatically generating inputs of death. In Proceedings of 13th ACM Conference on Computer and Communications Security, CCS'06. ACM, pp. 322–335.
Cadar, C., Godefroid, P., Khurshid, S., Păsăreanu, C. S., Sen, K., Tillmann, N. & Visser, W. (2011) Symbolic execution for software testing in practice: preliminary assessment. In Proceedings of 33rd International Conference on Software Engineering, ICSE'11. ACM, pp. 1066–1071.
Cadar, C. & Sen, K. (2013) Symbolic execution for software testing: Three decades later. Commun. ACM 56 (2), 8290.
Chamarthi, H. R., Dillinger, P. C., Kaufmann, M. & Manolios, P. (2011) Integrating testing and interactive theorem proving. In Proceedings of 10th International Workshop on the ACL2 Theorem Prover and its Applications, Electronic Proceedings in Theoretical Computer Science, vol. 70, pp. 4–19. http://www.eptcs.org/
Christiansen, J. & Fischer, S. (2008) EasyCheck – test data for free. In Proceedings of 9th International Symposium on Functional and Logic Programming, FLOPS'08, Lecture Notes in Computer Science, vol. 4989. Springer, pp. 322–336.
Claessen, K., Duregård, J. & Pałka, M. H. (2014) Generating constrained random data with uniform distribution. In Proceedings of 12th International Symposium on Functional and Logic Programming, Lecture Notes in Computer Science, vol. 8475. Springer, pp. 18–34.
Claessen, K. & Hughes, J. (2000) QuickCheck: A lightweight tool for random testing of Haskell programs. In Proceedings of 5th ACM SIGPLAN International Conference on Functional Programming, ICFP'00. ACM, pp. 268–279.
Clarkson, M. R. & Schneider, F. B. (2010) Hyperproperties. J. Comput. Secur. 18 (6), 11571210.
Dybjer, P., Haiyan, Q. & Takeyama, M. (2003) Combining testing and proving in dependent type theory. In Proceedings of 16th International Conference on Theorem Proving in Higher Order Logics, TPHOLs'03, Lecture Notes in Computer Science, vol. 2758. Springer, pp. 188–203.
Eastlund, C. (2009) DoubleCheck your theorems. In Proceedings of 8th International Workshop on the ACL2 Theorem Prover and its Applications, ACL2'09. ACM, pp. 42–46.
Fenton, J. S. (1974) Memoryless subsystems. Comput. J. 17 (2), 143147.
Fetscher, B., Claessen, K., Palka, M. H., Hughes, J. & Findler, R. B. (2015) Making random judgments: Automatically generating well-typed terms from the definition of a type-system. In Proceedings of 24th European Symposium on Programming, ESOP'15, Lecture Notes in Computer Science, vol. 9032. Springer, pp. 383–405.
Giffin, D. B., Levy, A., Stefan, D., Terei, D., Mazières, D., Mitchell, J. & Russo, A. (2012) Hails: Protecting data privacy in untrusted web applications. In Proceedings of 10th Symposium on Operating Systems Design and Implementation, OSDI'12. USENIX Association, pp. 47–60.
Godefroid, P., Klarlund, N., & Sen, K. (2005) DART: Directed automated random testing. In Proceedings of 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI'05. ACM, pp. 213–223.
Goguen, J. A. & Meseguer, J. (1984) Unwinding and inference control. In Proceedings of IEEE 1984 Symposium on Security and Privacy. IEEE CS, pp. 75–87.
Groce, A., Holzmann, G. J. & Joshi, R. (2007) Randomized differential testing as a prelude to formal verification. In Proceedings of The 29th International Conference on Software Engineering, ICSE'07. IEEE CS, pp. 621–631.
Guernic, G. L. (2007) Automaton-based confidentiality monitoring of concurrent programs. In Proceedings of 20th Computer Security Foundations Symposium, CSF'07. IEEE CS, pp. 218–232.
Guernic, G. L., Banerjee, A., Jensen, T. P. & Schmidt, D. A. (2006) Automata-based confidentiality monitoring. In Proceedings of 11th Asian Computing Science Conference, ASIAN 2006. Springer, pp. 75–89.
Hedin, D. & Sabelfeld, A. (2012) Information-flow security for a core of JavaScript. In Proceedings of 25th IEEE Computer Security Foundations Symposium (CSF), CSF'12. IEEE CS, pp. 3–18.
Hriţcu, C., Greenberg, M., Karel, B., Pierce, B. C. & Morrisett, G. (2013a) All your IFCException are belong to us. In Proceedings of 34th IEEE Symposium on Security and Privacy, SP'13, IEEE CS, pp. 3–17.
Hriţcu, C., Hughes, J., Pierce, B. C., Spector-Zabusky, A., Vytiniotis, D., Azevedo, de Amorim, A. & Lampropoulos, L. (2013b) Testing noninterference, quickly. In Proceedings of 18th ACM SIGPLAN International Conference on Functional Programming, ICFP'13. ACM, pp. 455–468.
Hughes, J. (2007) QuickCheck testing for fun and profit. In Proceedings of 9th International Symposium on Practical Aspects of Declarative Languages, PADL'07, Lecture Notes in Computer Science, vol. 4354. Springer, pp. 1–32.
Kinder, J. (2015) Hypertesting: The case for automated testing of hyperproperties. In Proceedings of 3rd Workshop on Hot Issues in Security Principles and Trust, HotSpot.
Klein, C. (August 2009) Experience with randomized testing in programming language metatheory. Master's Thesis, Northwestern. Available at: http://plt.eecs.northwestern.edu/klein-masters.pdf. Accessed Feb 26, 2016.
Klein, C., Clements, J., Dimoulas, C., Eastlund, C., Felleisen, M., Flatt, M., McCarthy, J. A., Rafkind, J., Tobin-Hochstadt, S. & Findler, R. B. (2012) Run your research: On the effectiveness of lightweight mechanization. In Proceedings of 39th ACM SIGPLAN-SIGACT Principles of Programming Languages, POPL'12, ACM, pp. 285–296.
Klein, C. & Findler, R. B. (2009) Randomized testing in PLT Redex. In Proceedings of Workshop on Scheme and Functional Programming, SFP, ACM, pp. 26–36.
Klein, C., Flatt, M. & Findler, R. (2013) The Racket virtual machine and randomized testing. In Higher-Order and Symbolic Computation, Springer, pp. 145. http://dx.doi.org/10.1007/s10990-013-9091-1
Koopman, P. W. M., Achten, P. & Plasmeijer, R. (2014) Model-based shrinking for state-based testing. In Proceedings of 14th International Symposium on Trends in Functional Programming, TFP 2013, Lecture Notes in Computer Science, vol. 8322, Springer, pp. 107–124.
Lampropoulos, L., Pierce, B. C., Hriţcu, C., Hughes, J., Paraskevopoulou, Z. & Xia, L. (July 2015) Making our own Luck: A language for random generators. Draft. https://www.cis.upenn.edu/~llamp/pdf/Luck.pdf
Leroy, X., Appel, A. W., Blazy, S. & Stewart, G. (June 2012) The CompCert memory model, version 2. Research report RR-7987, INRIA.
Leroy, X. & Blazy, S. (2008) Formal verification of a C-like memory model and its uses for verifying program transformations. J. Autom. Reason. 41 (1), 131.
Lindblad, F. (2007) Property directed generation of first-order test data. In Proceedings of 8th Symposium on Trends in Functional Programming, TFP'07, Trends in Functional Programming, vol. 8. Intellect, pp. 105–123.
Majumdar, R. & Sen, K. (2007) Hybrid concolic testing. In Proceedings of 29th International Conference on Software Engineering, ICSE'07. IEEE CS, pp. 416–426.
Milushev, D., Beck, W. & Clarke, D. (2012) Noninterference via symbolic execution. In Proceedings of Joint 14th IFIP WG 6.1 International Conference on Formal Techniques for Distributed Systems and 32nd IFIP WG 6.1 International Conference, FMOODS 2012 and FORTE 2012, Lecture Notes in Computer Science, vol. 7273. Springer, pp. 152–168.
Mohr, R. & Henderson, T. C. (1986) Arc and path consistency revisited. Artif. Intell. 28 (2), 225233.
Ochoa, M., Cuéllar, J., Pretschner, A. & Hallgren, P. (2015) Idea: Unwinding based model-checking and testing for non-interference on EFSMs. In Proceedings of 7th International Symposium on Engineering Secure Software and Systems, ESSoS'15, Lecture Notes in Computer Science, vol. 8978. Springer, pp. 34–42.
Pacheco, C. & Ernst, M. D. (2007) Randoop: Feedback-directed random testing for Java. In Proceedings of 22nd ACM SIGPLAN Conference on Object-Oriented Programming Systems And Applications, OOPSLA'07. ACM, pp. 815–816.
Paraskevopoulou, Z., Hriţcu, C., Dénès, M., Lampropoulos, L. & Pierce, B. C. (2015) Foundational property-based testing. In Proceedings of 6th International Conference on Interactive Theorem Proving, Urban, C. & Zhang, X. (eds), ITP'15, Lecture Notes in Computer Science, vol. 9236. Springer, pp. 325–343.
Regehr, J., Chen, Y., Cuoq, P., Eide, E., Ellison, C. & Yang, X. (2012) Test-case reduction for C compiler bugs. In Proceedings of 33rd ACM SIGPLAN conference on Programming Language Design and Implementation, PLDI'12. ACM, pp. 335–346.
Runciman, C., Naylor, M. & Lindblad, F. (2008) SmallCheck and Lazy SmallCheck: Automatic exhaustive testing for small values. In Proceedings of 1st ACM SIGPLAN Symposium on Haskell. ACM, pp. 37–48.
Russo, A. & Sabelfeld, A. (2010) Dynamic versus static flow-sensitive security analysis. In Proceedings of 23rd Computer Security Foundations Symposium, CSF'10. IEEE CS, pp. 186–199.
Sabelfeld, A. & Myers, A. (January 2003) Language-based information-flow security. IEEE J. Sel. Areas Commu. 21 (1), 519.
Sabelfeld, A. & Russo, A. (2010) From dynamic to static and back: Riding the roller coaster of information-flow control research. In Proceedings of 7th International Andrei Ershov Memorial Conference, PSI 2009, Lecture Notes in Computer Science, vol. 5947. Springer, pp. 352–365.
Sabelfeld, A. & Sands, D. (2005) Dimensions and principles of declassification. In Proceedings of 18th IEEE Workshop on Computer Security Foundations, CSF'05. IEEE CS, pp. 255–269.
Sen, K., Marinov, D. & Agha, G. (2005) CUTE: A concolic unit testing engine for C. In Proceedings of 10th European Software Engineering Conference held jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering, ESEC/FSE-13. ACM, pp. 263–272.
Stefan, D., Russo, A., Mitchell, J. C. & Mazières, D. (2011) Flexible dynamic information flow control in Haskell. In Proceedings of 4th Symposium on Haskell. ACM, pp. 95–106.
Stefan, D., Russo, A., Mitchell, J. C. & Mazières, D. (July 2012) Flexible dynamic information flow control in the presence of exceptions. ArXiv e-print 1207.1457.
Terauchi, T. & Aiken, A. (2005) Secure information flow as a safety problem. In Proceedings of 12th International Symposium on Static Analysis, SAS 2005, Lecture Notes in Computer Science, vol. 3672. Springer, pp. 352–367.
Torlak, E. & Bodík, R. (2014) A lightweight symbolic virtual machine for solver-aided host languages. In Proceedings of ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2014. ACM, 2014, pp. 530–541.
Williams, N., Marre, B. & Mouy, P. (2004) On-the-fly generation of K-path tests for C functions. In Proceedings of 19th IEEE International Conference on Automated Software Engineering, ASE. IEEE CS, pp. 290–293.
Yang, X., Chen, Y., Eide, E. & Regehr, J. (2011) Finding and understanding bugs in C compilers. In Proceedings of 32nd SIGPLAN Conference on Programming Language Design and Implementation, PLDI'11, ACM, pp. 283–294.
Zdancewic, S. A. (2002) Programming Languages for Information Security. PhD Thesis, Cornell University.
Zeller, A. & Hildebrandt, R. (2002) Simplifying and isolating failure-inducing input. IEEE Trans. Softw. Eng. 28 (2), 183200.
Zheng, L. & Myers, A. C. (2007) Dynamic security labels and static information flow control. Int. J. Inform. Secur. 6 (2–3), 6784.

Related content

Powered by UNSILO

Testing noninterference, quickly

  • CĂTĂLIN HRIŢCU (a1), LEONIDAS LAMPROPOULOS (a2), ANTAL SPECTOR-ZABUSKY (a2), ARTHUR AZEVEDO DE AMORIM (a2), MAXIME DÉNÈS (a3), JOHN HUGHES (a4), BENJAMIN C. PIERCE (a2) and DIMITRIOS VYTINIOTIS (a5)...

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.

Testing noninterference, quickly

  • CĂTĂLIN HRIŢCU (a1), LEONIDAS LAMPROPOULOS (a2), ANTAL SPECTOR-ZABUSKY (a2), ARTHUR AZEVEDO DE AMORIM (a2), MAXIME DÉNÈS (a3), JOHN HUGHES (a4), BENJAMIN C. PIERCE (a2) and DIMITRIOS VYTINIOTIS (a5)...
Submit a response

Discussions

No Discussions have been published for this article.

×

Reply to: Submit a response


Your details


Conflicting interests

Do you have any conflicting interests? *